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^ ! Abstract 

In this paper, we explore the mystery of synchronous CDMA as applied to wireless and optical 
communication systems under very general settings for the user symbols and the signature matrix entries. 
The channel is modeled with real/complex additive noise of arbitrary distribution. Two problems are 
addressed. The first problem concerns whether overloaded error free codes exist in the absence of additive 



ryl ' noise under these general settings, and if so whether there are any practical optimum decoding algorithms. 

The second one is about the bounds for the sum channel capacity when user data and signature codes 



employ any real or complex alphabets (finite or infinite). In response to the first problem, we have 
developed practical Maximum Likelihood (ML) decoding algorithms for overloaded CDMA systems for 
a large class of alphabets. In response to the second problem, a general theorem has been developed 



> 

^D \ in which the sum capacity lower bounds with respect to the number of users and spreading gain and 



Signal-to-Noise Ratio (SNR) can be derived as special cases for a given CDMA system. To show the 
^^ . power and utility of the main theorem, a number of sum capacity bounds for special cases are simulated. 

An important conclusion of this paper is that the lower and upper bounds of the sum capacity for 
small/medium size CDMA systems depend on both the input and the signature symbols; this is contrary 
to the asymptotic results for large scale systems reported in the literature (also confirmed in this paper) 
where the signature symbols and statistics disappear in the asymptotic sum capacity. Moreover, these 
questions are investigated for the case when not all users are active. Furthermore, upper and asymptotic 
bounds are derived and numerically evaluated and compared to other derivations. 

Manuscript received 

K. Alishahi is affiliated with Advanced Communication Research Institute (ACRI) and Dept. of Mathematical Sciences, Sharif 
University of Technology. 

S. Dashmiz, R Pad, R Marvasti, M. H. Shafinia and S. M. Mansouri are affiliated with ACRI and Electrical Engineering 
Department, Sharif University of Technology marvasti@sharif.edu. 



I. Introduction 

Code Division Multiple Access (CDMA) has become the standard for the third generation of Global 
System for Mobile (GSM) communications and American cellular systems [1]. Also, optical CDMA 
systems have become an alternative multiple access for fiber optics and optical wireless systems [2]-[4]. 
The reasons, to name a few, are its simplicity, high overloading factor and soft hand-off. On the other 
hand, the theoretical developments in CDMA sum capacity have been limited to users with Gaussian 
inputs where Welch Bound Equality (WBE) codes achieve the theoretical capacity [5]-[9]. For binary 
CDMA, only asymptotic results were known [10]-[13] prior to our recent papers [14], [15] (to be 
discussed in the next paragraph). For overloaded CDMA, only WBE sequences were known to minimize 
the user interference using autocorrelator detectors and iterative methods [16]-[28]. Simple ML decoding 
for binary overloaded CDMA and its superiority to WBE sequences were shown in [14], [29]-[31]. ML 
decoding of general user inputs and signature matrices has not been reported in the literature, nor has 
the evaluation of the sum capacity for such general CDMA systems. The revisitation and the solutions 
of such CDMA problems are the main topics of the current paper. 

For binary synchronous CDMA systems, a class of errorless codes for overloaded CDMA systems were 
introduced [14], [32]; also in [15], we derived a relatively tight family of bounds for the sum capacity of 
binary CDMA systems with binary signatures. The present paper is a continuation of the previous ones; 
the constraints of binary multi-user and binary signatures are now relaxed. The errorless codes (COWl^ 
and COCp) developed in [32] and [14] are also extended to nonbinary cases. We have also extended the 
results to active user detection in a CDMA system. Below, we will give a brief summary of the previous 
results: 

In [14], [29], [32], a class of overloaded errorless codes for binary multi-users and binary/ternary 
signatures for wireless (±1 for COW) and optical (0, 1 for COO) applications were developed. Mow [32] 
presented a unifying approach to find one-to-one mxn binary and ternary matrices for binary inputs for 
multiuser applications. He also applied constructive theorems developed by the previous authors [33]- 
[39] to enlarge such matrices. This paper also discusses asymptotic behavior of such matrices. In [14], 
we have also developed injective binary matrices for binary inputs independently. In the same paper, we 
have also suggested ML decoding using tensor products. 

In addition, in [14], we developed lower and upper bounds for the sum capacity of a binary synchronous 

'Codes for Overloaded Wireless systems 
^Codes for Overloaded Optical systems 



CDMA under noiseless environment conditions and the assumption that users are jointly dependent. These 
sum capacity bounds suggest that there exists a linear region with respect to the number of users far beyond 
the orthogonal Walsh codes (n = m where m is the chip rate), i.e., errorless overloaded (n » m) codes 
do exist. The bounds also suggest that there is a threshold beyond which overloaded errorless codes do 
not exist. In the same paper, we developed practical COW/COO codes (n = 104, m = 64) with simple 
Maximum Likelihood (ML) decoders that outperformed Welch Bound Equality (WBE) codes both in 
terms of complexity and performance. WBE codes are optimum (capacity achieving) for multi-users of 
Gaussian real inputs but not for binary users. 

In [15], we extended the bounds to a noisy channel with arbitrary distribution. We also changed the 
assumption of joint probability of multi-users to a more realistic scenario of independence among users. 
For the noiseless case, the same bounds were derived as the ones in [14] despite the independence 
assumption. Simulation results again showed that there is a linear region beyond Hadamard codes and 
there is a degraded region for a highly overloaded CDMA. The asymptotic results for the sum capacity 
simplified the equations and thus the computer simulations, and we managed to compare our results 
to that of Tanaka [10]. The main contribution of [15] are tight bounds with closed form derivations 
that, unlike the previous papers, depend not on the limiting cases but rather on the number of users n, 
spreading gain m, and the noise distribution that need not be Gaussian. 

In the present paper, we will extend the binary results to multi-users and signature matrices with 
entries from finite/infinite, real/complex alphabets. Extensions of overloaded codes (COW and COO) to 
nonbinary CDMA and active user detection are also considered. An important achievement in this regard 
is the development of practical overloaded codes with optimal low complexity decoding algorithms. 

Bounds for both noisy and noiseless channels are considered. Our main achievement in this regard is 
the development of a general theorem for the lower bounds where any CDMA system with arbitrary user 
data and signature matrix symbols can be modeled as a special case. We further extend the sum capacity 
bounds to k-active users where not all users transmit data. 

Some of the main results and theorems wiU be reviewed in the next section on preliminaries; the 
structure of the paper is as shown belowcl: 

Contents 
I Introduction 

^The table of contents is for the convenience of the reviewers and will not be in the final galley proof. 
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In the section on preliminaries, four topics are reviewed: 1- power constraint considerations, 2- the 
definition of sum capacity and the main known results, 3- review of the COW/COO matrices, and 4- 
review of the sum capacity bounds for the binary CDMA. Section |lll] deals with the generalization of the 
COW/COO codes to the nonbinary finite real/complex CDMA systems (GCO). Practical algorithms for 
the reconstruction of GCO matrices and bounds on the overloading factor are introduced in this section. 
The performances of GCO matrices, as compared to the WBE codes, and practical ML decoders are 
covered in this section. Active user detection for CDMA systems is considered as a special case of GCO 
matrices and is discussed at the end of this section. 

Section |IV] covers the derivation of lower bounds for the sum capacity of an arbitrary CDMA system 
with no channel noise; a general theorem is developed and special cases of finite {q, /)-ary CDMA systems 
are developed. Many examples such as binary/ternary and binary/quaternary CDMA systems are derived 
with simulation results. A presentation of upper bounds and asymptotic cases is the final parts of this 
section. 

Section |V] is similar to the previous section except additive channel noise with arbitrary distribution 
is also considered; the noiseless case of Section |IV] can be derived from the noisy case in the limiting 
case. In this section, we will develop a general theorem for the lower bound that is valid for any type 
of CDMA system; from this theorem, we shall develop in the subsequent two sections, the special cases 



for various types of input and signature symbols. In SectioifVll we will derive the lower bound from 
our main theorem for the special cases of finite {q, /)-ary and finite/real-complex CDMA systems. As 
special cases, some examples are simulated for the binary/ternary, ternary/ternary, and binary/real CDMA 
systems with additive Gaussian noise. Upper bounds and asymptotic limits are also derived in the same 
section. 

Section rvm is related to the bounds for the case when the number of active users is either deterministic 
or random. For the deterministic case, special cases of binary/real, real/real, and binary/binary are also 
formulated and simulated. For the random case, similar results are derived. The asymptotic bound for 
the random case is also considered. 

Finally, a summary of the main results, concluding remarks, and future studies are covered in Section 
IVIIII All the proofs of the theorems and examples are provided in the appendices. 

II. Preliminaries 

Consider a CDMA system with n users and m chips as y = -^AX + N, where X is the n x 1 user 
data vector, N is the m x 1 noise vector, A is the m x n signature matrix, and Y is the m x 1 received 
signal (these vectors and matrices could be real or complex). 

A. Power Constraint Issues 

Since the total power of users and noise in the above model are, respectively, equal to 
tr (E (^AXX*A*)) and E(A^*A^) (the symbol * stands for the Hermitian transpose), the multi-user 
SNR is defined as 

tr(E(^AXX*A*)) 
SNR = ^ ^"^, ^ ^. (1) 

For simplicity, we will assume that N = [Ni,N2, ■ ■ ■ ,Nm] , where A'^'s are i.i.d. random variables 
with /(•) as common Probability Distribution Function (pdf) of variance a'j. This assumption implies 

,2 



that E {N*N) = ma i and hence the SNR definition can be written as 



tr(E(^AXX*A*)) 
SNR = ^ ^"" 2 ■ (2) 



If —SNR < rj, the above equation can be written as 

tr [ E l—AXX*A*\\ < nrjaj. (3) 

For a given signature matrix -^A and rj, the sum capacity is defined as 

C {A,ri) = max{I{X;Y) \X ~ pi (xi) x p2 (^2) x • • • x p„ (x„) such that © is satisfied}. (4) 



However, we would like to maximize C (A, rf) over a class of admissible signature matrices -^ A 
and input vectors X as follows: 
Let S and X be two given sets of real or complex numbers which could be finite or infinite. Assume that 

A = rB rEM+, B G TW^xn (5) (5) 

and 

XG7W„xi(X), (6) 

where r is the scale factor that guarantees the power condition in ([3]). With the assumption that Xj's are 
i.i.d. with distribution p{-), we have E {XX*) = a^ln + fJ-p^Jn, where fip and ap denote the mean and 
variance; I„ and J„ are, respectively, the n x n identity matrix and the matrix with all entries equal to 
one. Therefore, we have 

tr (e (—AXX*aAj = —{ap'^tr{BB*) + fiphr{BJB*)). (7) 

Hence the SNR condition ^ can be written in the following form 

2 

— {a'^tr{BB*) + Hp^ti{BJB*)) < nr]aj. (8) 

B. Definition of Sum Channel Capacity and the Main Known Relevant Results 

Definition 1 (The Sum Channel Capacity): 
Define C {171,11,1,3,7]) to be the maximum value of C(A,r/) in Q over all signature matrices -^A, 
where ^, ^ and ([8]l are satisfied. For noiseless systems, r] = oo, we use the notation C {m,n,X,S). 

Remark 1: References [10], [19], [40] have used the average mutual information over all signature 
matrices as the sum capacity. Such interpretation is actually less than the sum capacity as defined in the 
above. In the present paper, our upper bounds are naturally the upper bounds for the average mutual 
information. Due to the techniques we use, the lower bounds derived in the present paper have smaller 
values than the average mutual information. 

Given the above definition, we are now ready to review the main relevant results from the CDMA 
literature: 

Theorem 1 (Main Known Result for the Sum Channel Capacity): 
When the input vector X is Gaussian, the signature matrix -^ is arbitrary, and the additive noise 
is white Gaussian, the sum channel capacity can be shown to be [5] 



C = - log ( det ( I + — AA' ) ) , for Real System (9) 



C = log (det (l + — AA*) ) , for Complex System, (10) 

where rj is the normalized SNR as defined in ([3]). 

Example 1 (The Sum Channel Capacity for WBE Matrices): 
Here we consider real WBE signature matrices since they maximize the capacity in Q, [7]. For un- 
derloaded case (n < m) and for the normalized WBE matrices, ^ becomes 

C = -log(l + 2r?), (11) 

and likewise for the overloaded case (n > m), ^ becomes 

^ m ( 2n \ 

C=-log 1 + — 77 . (12) 

2 V m J 



Theorem 2: Tanaka's Asymptotic Bound for Binary User Input 
For binary bipolar input and binary matrix, Tanaka derived an asymptotic bound using the replica theory 
in statistical physics [10]. 

in which 



Ce = — log(l+ ^^ 2 ^ ]+9{\0)log{e), (13) 



g{X,e) = ^il + e)- f In [cosh [^/XZ + X^^Dz, (14) 



where Dz is the standard normal measure and 

9= f tanh (VXZ + x]Dz , (16) 

where B = — and a^ is the variance of the noise. 

1^ m 

We have shown in [15] that (IT3] ) approaches our asymptotic upper bound when /3 increases. The replica 
theory is a non-rigorous mathematical analysis. A rigorous proof of Tanaka's results is given in [11], 
[40], [41]. The extension of Tanaka's bound to arbitrary input symbols is given in [12]; for complex 
input symbols, the following asymptotic result can be derived: 

Theorem 3: Guo-Verdu Asymptotic Bound for Arbitrary User Input [12] 

Cjo^nt (/3) = /3E{I {r,' s) } + {ti'- 1) log e - log t]' , (17) 



in which 

I (77' s) = B {pz\x,.,^'\\Pz\.,^'\px) , (18) 

where /3 = n/m, s is the single user SNR, rj' is the multiuser efficiency determined from recursive 
relationships that depend on the type of detectors and Z = ^/sX + -M=. 

It is also shown in [40] that the asymptotic bound does not depend on the distribution of the signature 
matrix. In this paper, we will show that for the noiseless case, the asymptotic lower bound depends on the 
input and signature symbols; however, for the noisy case, the asymptotic lower bound does not depend 
on the signature matrix symbols. 

Since most of the results of [14] and [15] are needed in the present paper, a summary of the results 
are also given here in the preliminaries. 

C. Summary of [14]: A Class of Errorless Codes for Overloaded Synchronous Wireless and Optical 
CDMA Systems 

[14] is on the construction and simple ML decoding of errorless binary codes in the absence of noise 
and without near-far effects for a synchronous CDMA system. The main results of this paper that will 
be needed in the present paper are the following theorems: 

Theorem 4 (Enlarging COW Matrices): 
Assume that Ccow is an tti x n COW matrix and P is an invertible k x k {±1} matrix, then Yicow = 
P (g) Ccow is a km x kn COW matrix, where ® denotes the Kronecker product. 

Moreover the decoding of a system with the code matrix Ticow can be reduced to k decoding systems 
with the code matrix Ccow, where the decoding of Ccow can be implemented by 2"^™ Euclidean distance 
measurements using ML decoding [30]. 

Theorem 5 (Highly Overloaded COW Matrices): 
Assume Ccow is an 771 X n COW matrix and H2 is a 2 x 2 Hadamard matrix. We can add \{m — 1) log3 2] 
columns to H2 ® Ccow to obtain another COW matrix. 



Theorem 6 (Upper Bound for the Overloading Factor): 
For a COW/COO matrix, the total number of users n is upper bounded by 



where (") 



il{n—i)l ' 



D. Summary of [15]. -Bounds on the Sum Capacity of Synchronous Binary CDMA Channels 

Theorem 7 (Noiseless Lower Bound): 
For any m and n, the sum capacity is lower bounded by 

C(m,n,{±l},{±l})>n-log^(^2^^.) (i|/j . (20) 



Theorem 8 (Noisy Lower Bound): 
In Additive White Gaussian Noise (AWGN) channel with the real noise of variance a^, rj = ^ is 
equivalent to the normalized SNR, Eb/NQ. For any m and n and any positive real number 7, the sum 
capacity, as described in Definition [H is lower bounded by: 

fc=o ^^ \i=o 



C{m,n,{±l},{±l},v)>n-mjlogV^-log^iU^^ -== . (21) 



Theorem 9 (Conjectured Noisy Upper Bound): 
If the noise pdf /(•) is symmetric, we have 

C (m, n, {±1}, {±1}, r?)< mill (n,m(]H(/)-H (/))), (22) 

where 

Theorem 10 (Asymptotic Noiseless Lower Bound): 
For large scale systems when .^^" is kept constant, the bound in (l20l ) becomes 

lim -C(m,n,{±l},{±l}) =min( 1,— ) . (24) 

n/(mlogn)-5>i n \ 2l J 

m,n— ^00 

Theorem 1 1 (Asymptotic Noisy Lower Bound): 
Likewise, for large scale systems when /3 = ^ is kept constant, the bound in (|2T] | becomes 



10 



lim -C{m,n,{±l},{±l},T]) 

m.n—^oo 

>supfl- sup [«(*)- i5l„,i±M^!±i^|), p5, 



where H{t) = -tlog(t) -{l-t) log(l - t). 



Theorem 12 (Asymptotic Noisy Upper Bound): 
Similarly, the bound in (l22l ) becomes 



lim ic (m, n, {±1}, {±l},r?)< mill f 1,4 log (l + 2r//3)). (26) 



m,n—^oo 



Note 1: Numerical Improvements of the Bounds for Overloaded Binary CDMA 
The bounds developed from Theorems [81IT2] for the binary CDMA systems are less accurate for under- 
loaded (small /3) and small scales {m, n). However, for /3 < 1, the Hadamard signature matrix is equivalent 
to the single user Binary Phase Shift Keying (BPSK) and its actual capacity is known analytically and thus 
can be simulated. Fig. [T] shows the actual normalized sum capacity based on both hard and softdesition 
(soft decision is identical to hard for r/ = 12 dB) for /3 < l,?7i = 8. The COW/COO overloaded codes 
behave similar to the Hadamard codes that are for the fully/underloaded CDMA systems. If the noise 
level is less than a certain amount, the probability of error can be determined based on hard decision ML 
decoding, and the actual normalized sum capacity can be simulated. The same figure shows the capacity 
of a specific COW matrix for /3 in the range (1, 13/8) for a signal-to-noise ratio i] of 12 dB. Clearly, 
we can have better numerical bounds than that of Theorem [8] 

Below, we will generalize the COW/COO codes of subsection III-Cl to nonbinary cases. The generalization 
of such codes for active users are also given in the same section. The rest of the sections are generalizations 
of the binary sum capacity bounds given in subsection III-DI for finite/infinite, and real/complex symbols 
including the near-far effects. 
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Fig. 1 . Normalized upper and lower bounds for the sum capacity derived from l l21b and l l22b and the actual noormalized hard 
decision capacity. For m = 8, r] — 12 (dB) and j3 > 1, the computer simulated sum capacity for the overloaded COW matrix 
is a better lower bound than the one predicted analytically from l l21t . 



III. Generalized COW/COO (GCO) Matrices 

In this section, we extend the class of COW/COO codes discussed in Subsection lll-Cl when the alphabets 
of the signatures and the user inputs are not limited to binary sets. In the following, we will discuss the 
construction of GCO matrices and the ML decoding of such codes. The concepts of fairness and active 
user detection will be examined. 



A. Constructing GCO Matrices 

Suppose that the inputs are from a given set I and the signature entries are from a given set S. For 
errorless communications, we need to find m x n signature matrices that are one-to-one over the set 
X". We call such matrices GCO {m,n,I,S). The main problem is to find values of m and n such that 
GCO (?7i, n,X, 5) matrices exist. In the following, we will develop a general theorem related to GCO 
matrices. Consequently, we will give corollaries and examples to construct and decode GCO matrices 
for overloaded nonbinary CDMA systems. In this section, we define I„i to be the first m — 1 columns of 
the m X m identity matrix I^, Jm to be the first m — 1 columns of the m x m all one matrix J^, K^ 
to be 2Im — Jm and Om to be the first m — 1 columns of the m x m zero matrix Om- We can generalize 
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Theorem |4] for enlarging COW matrices to GCO matrices as shown in the following theorem: 

Theorem 13 (Constructing Larger GCO Matrices from Smaller Ones): 
Assume that A is a GCO {m,n,X,S) matrix and P is a tD x w invertible matrix with entries belonging 
to the set IZ. Then, P A is a GCO {mw, nw,I, S- IZ) matrix where S-TZ is, the set of all products of 
the elements of S and TZ. 

The proof is very similar to that of Theorem |4] in the preliminaries section, which is given in [29]. 
A generalization of Theorem |5] to GCO matrices is given below: 

Theorem 14 (A General Method for Constructing GCO Matrices): 
For integer sets of X, 5, assume that C is a GCO (m,n,X, 5), H^ is a it; x w Hadamard matrix and 
k is an integer number. Also, suppose that ±.kS C S' and d is the largest non-zero integer such that 
^^ C Z. If Ml, M2, . . . , Mi„ are matrices with entries from S' such that X^iLi ^j = ^^m, where t is 
any integer, then 



B 



Ml 
M2 
k'Hyj (g) C . 

M„ 



(27) 



is a GCO {wm, wn + m,I, S') matrix if 



^^^:^G4n^ = {0}. (28) 



t J d 

moreover, when S consists of only odd numbers and J27=i ^i — ^^nn then B is 

GCO {wm, wn + m — 1,1, S') matrix if 

(2uwk 1 X-X 

}——:ueZ\n^^ = {0}. (29) 

The proof is given in Appendix [A] 

Corollary 1 (Constructing Binary GCO Matrices): 
Assume that A is a GCO {m, n, {0, ±1, . . . , ±g}, {±1}) matrix, H4r is a 4r x 4r Hadamard matrix and 
4r >2q + l. Also, assume that Mi = J™, M2 = Km, Mj = J^ for i = 3,4, . . . ,2r + l and M, = -J^ 
for i = 2r + 2, 2r + 3, . . . , 4r. Now, 
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is a GCO {Arm, Am + m — 1, {0, ±1, . 
The proof is given in Appendix |B] 



Ml 
M, 



H4r (g) A 

M4,.. 

. , ±g}, {±1}) matrix. 



(30) 



Example 2 ((3,2)-ary GCO): Table U is a GCO (16, 22, {0, ±1}, {±1}) matrix which is about 38% 
over-loaded [29]. Using the above theorem, we get a GCO (64, 103, {0, ±1}, {±1}) matrix which is 
about 60% over-loaded. Notice that the first 16 columns of the above matrix is a 16 x 16 Hadamard 
matrix. 

TABLE I 

Ai6x22 WHERE + DENOTES +1 AND - DENOTES -1. 



++++++++++++++++++ 

- + - + - + - + - + - + - + + 

+ + + + + + + + 

+ + + + + + + + + - 

+ + + + + + + 

- + + - + + - + + - + + - + 

H + + + H 1-+-+ + 

+ - + + - + + - + + 

+ + + + + + + + 

- + - + - + + - + - + - + + - + 

+ + + + + + + + + + 

++ + - + + + + + + 

+ + + + + + + + + - 

- + + - + - + - + + - + - + - + 

+ + + + + + + + 

1 hH I- + -H 1- + + + 



Example 3 ((7,2)-ary GCO): Note that Hs is a GCO (8, 8, {0, ±1, ±2, ±3}, {±1}) matrix. Using the 
above theorem, we get a GCO (64, 71, {0, ±1, ±2, ±3}, {±1}) matrix. 

Example 4 ((2,2)-ary GCO): Using theorem [I4l one can start with a 2 x 2 Hadamard matrix and 
construct a GCO (64, 193, {±1}, {±1}) matrix which is about 200% over-loaded. This example is similar 
to that of [32], [42]. 



Example 5 ((2,3)-ary GCO): Suppose A is a GCO (?7i, n, {±1}, {±1}); from Theorem [T4l by setting 
k = 1, w = I, S = {±1}, S' = {0, ±1} and Mi = I^, we get i = 1 and d = 2. One can easily check 
that 



( 2uwk ^ I~I ,^, 

|^:nez|n^ = {0}. 



(31) 
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Thus [A Ml ] is a GCO (m, m + n — I, {±1}, {0, ±1}) matrix. If we choose A to be the 

GCO (64, 193, {±1}, {±1}) matrix generated from Example IH we get a 

GCO (64, 256, {±1}, {0, ±1}) matrix, which is 300% overload. This example is also similar to that of 

[32]. 



Corollary 2 (Constructing Nonbinary GCO Matrices): 
Assume that A is a GCO (m, n, {0, ±1, . . . , ±q}, {0, ±1, . . . , ±1}) matrix and H^j is a wxw Hadamard 
matrix. Also, assume that Mi = I^ and Mj = 0^ for i = 2, 3, . . . , t(;. Now, 



/jH„ 



Ml 
M, 



M„ 



(32) 



is a GCO [mw, nw + m, {0, ±1, . . . , ztg}, {0, ±1, . . . , -^Ik}) matrix, if wk>2q. 
The proof is similar to that of Corollary [T] 



Example 6 ((3,3)-ary GCO): It is easy to verify that the matrix shown in Table HIl is a 
GCO (4, 6, {0, ±1}, {0, ±1}) matrix. Applying the previous theorem two times, we obtain a 
GCO (64, 128, {0, ±1}, {0, ±1}) matrix. 

TABLE II 

A4x6 WHERE " + " DENOTES +1 AND " — " DENOTES —1. 



+ + + + + 

+ - - + 

+ - + - 

+ + - - 



Example 7 ((3,5)-ary GCO): From the previous example, we conclude that there exists a 
GCO (64, 128, {0, ±1}, {0, ±1}) matrix. Applying the previous theorem , we obtain a 
GCO (128, 320, {0, ±1}, {0, ±1, ±2}) matrix. 
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Theorem 15 (Generation of Overloaded Complex GCO Matrices for Arbitrary Input Integers): 
Let X be a finite set of integers and assume that Aj is a GCO (m, n,X, {±1}) matrix for i = 0, 1, 2, ... , k, 
and {1, ^1, ^2; • • • , ^fc} are k+1 AESO (real or complex) numbers. Also, assume that Bj = \iiw ® Aj M*] 



/MiA 



where M* 



are GCO (m,n,X, {0, ±1}) matrices generated as demonstrated in Theorem [141 

VM^7 

for i = 1, 2, . . . , /j. Suppose Dj = H^ (g) Aj for i = 0, 1, 2, . . . , /c and M* is obtained by replacing each 

entry of ^jM* by +1 or —1 arbitrarily. Then, 

B = [Do eiDi 6D2 ... ^Dfe Mi...M'=]isa 

GCO {mw,{k + l)nw + (m - 1) A;, {±1}, {±1, ±^1, . . . ,±ik}) matrix. 

The proof is given in Appendix O 

Corollary 3 (Binary Input with Complex Matrix GCO): 
Assume that Aj's are GCO (m, n, {±1}, {±1}) for i = 0, 1, 2, . . . , A; — 1, and ^ is a complex number 
with minimal polynomial of degree k. Also, assume that Mj = — Jm + (l + C*) Im for i = 1, . . . , /c — 1. 
Then, [Ao ^Ai ... ^-^J^k-i Mi ... M^.J is a 
GCO (m, kn + {m- 1) (/c - 1) , {±1}, {±1, ±^, . . . , ±i^~^}) matrix. 
The proof is straightforward from Example [5] and Theorem [15] 

Example 8 (Binary Input with Quaternary Complex Matrix): For the special case of ^ = e ^ ,m = 64, 
n = 193, A; = 2, we get a GCO(64, 449, {±1} , {±1, ±j}). But in [43] they achieved an overloaded 
GCO(64, 128, {±1} , {±1, ±j}), which is much smaller than the matrix derived from the above theorem. 
Now that we have given examples of constructing GCO codes, we can discuss the upper bound for the 
number of users and practical and optimum decoding of such codes. 

B. Upper Bound for the Overloading Factor 

The following theorem provides a general upper-bound for the overloading factor for GCO matrices. 



''We call a set of numbers Algebraically Independent Numbers (AIN), if linear combinations of the numbers with integer 
coefficient cannot become zero. 



16 



Theorem 16 (A General Upper Bound for GCO{m,n,X,S) Matrices): 
If there exists a GCO{m,n,Z,S), then 

nlog2|2^|<w. max El(yi), (33) 

(ai,...,a„)e5" 

where P(Yi = I) is equal to the number of solutions of equation X]?=i '^j-^j = I viiZ, divided by |X|". 
The Proof is given in Appendix iDl 

Example 9 (Binary and Ternary Cases): 
For Z = S = {±1}, the upper bound (1331 ) becomes identical to that given in ( fT9l ). On the other hand, 
for the special case Z = {0, ±1} and S = {±1}, we get the following upper bound [29]. 

^<_^l^__log2__l, (34) 

where 

The above upper bound is simulated in Fig. |2l This figure shows that we cannot have errorless commu- 
nications beyond 230 ternary users (equivalent to 230 log2 3 ss 365 bits) with 64 chips. It is interesting 
to compare this number to the binary case in ( fT9l ) where the upper bound for the number of users is 268. 



C. ML Decoding for a Class of GCO Codes 

The direct ML decoding of GCO codes is computationally very expensive for moderate values of m 
and n. In this subsection, we present two lemmas for decreasing the computational complexity of the 
ML decoding for a class of GCO codes. 

Lemma 1 (Decoding Method for Large Matrices): 
Suppose 'Dwmy^wn = P«>x«> ® ^mxn IS Constructed as discussed in Theorem [T3l The decoding problem 
of a system with the code matrix D can be reduced to k decoding problems of a system with the code 
matrix A. 

Proof: The proof is similar to that of Theorem |4] in the preliminaries. 
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Fig. 2. Upper bound on the maximum number of overloaded users (nth) vs. the number of chips for a (3, 2)-ary CDMA 
system in the absence of additive noise. 



Lemma 2 (Optimum Decoding Algorithm): 
If A is a full rank GCO (m,n,X, 5), tlien tiie decoding problem for a system with the code matrix 
A can be performed through \I\'^~'^ Euclidean distance measurements. 

Proof: The proof is similar to that of Theorem |4] in the preliminaries; however, instead of using the 
sign function, a vector is mapped to the nearest X-vector. 

Lemmas [U and [2] lead to significant decrease of decoding complexity. Since each vector is mapped to 
the nearest Z vector, it is not hard to show that if m columns of A make a scaled unitary matrix, then 
the proposed method for decoding is an ML decoder. 



D. Simulation Results for GCO Matrices 

Figure [3] shows the performance of a GCO matrix (3, 2) generated from the kronecker product of the 
matrix represented in Table U by a 4 x 4 Hadamard matrix in a noisy environment. This GCO matrix has 
dimension 64 x 88; it also confirms that GCO codes, similar to COW/COO codes for the binary case. 



18 



are superior to WBE sequences. The ML decoding has been performed by using Lemmas \T\ and |2l 
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Fig. 3. Symbol-Error-Rate versus normalized SNR (rj) for an overloaded CDMA system with 64 chips and 88 users for a 
(3,2)-ary GCO system with signature matrix H4 ® Ai6x22 (Table [ill. 



Note 2: Fairness 

The COW/COO codes, for a given signature matrix, do not have identical performance in the presence 
of additive noise. The performance of (m = 64,n = 96) COW codes is shown in Fig. |4l On the same 
figure, one can see (64,96) Binary WBE (BWBE) codes [44] for comparison. Although the BWBE code 
looks fairer, its performance is not acceptable. On the other hand, the Hadamard codes for underloaded 
CDMA and the Optical Orthogonal Codes (OOC) are fair, but OOC codes have poor performance for 
the overloaded case. 

The extension of the concept of fairness to the general case is also simulated in Fig. |5] for a 
GCO (64, 88, {±1}, {0, ±1}). This figure also shows that the GCO codes are not as fair as WBE codes 
but their performances are better than that of WBE codes. 
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Fig. 4. Bit Error Rate (BER) versus i] for different code signatures for specific COW and BWBE matrices. The COW codes 
are not as fair as BWBE codes but their performances are better than that of BWBE codes. 



E. Codes for Overloaded Wireless CDMA with Detection of Active Users (COWDA) 

In a practical CDMA system, not all users are always active. This can be modeled by ±1 for binary 
user data and when nothing is senjj. This becomes problematic for Multi User Detection (MUD); to 
obtain errorless communication, we need binary signature matrices that are one-to-one over ternary input 
vectors. This class of codes are named Codes for Overloaded Wireless CDMA with Detection of Active 
(COWDA) users. This class is identical to GCO (m, n, {0, ±1}, {±1}) as discussed in Section Hill 

1) Decoding Algorithm and Simulation Results: The decoding algorithm is identical to that of Sub- 
section IIII-CI This decoding has the added advantage that it works with an unknown number of active 
users [29]. 

To show the behavior of COWDA codes with respect to additive noise, we have simulated a CDMA 
system with 64 chips and 88 users in the presence of AWGN. The BER versus 77 is depicted in Fig. 
|6] The simulation result for the COWDA code is obtained by using input vectors X with equiprobable 
entries in the set {0, ±1}. This is not a typical case and does not occur in practice because, in actual 
systems, when a user is active, it stays active for a period of time. Thus, entries at different times are not 
independent and do not randomly alternate between ±1 and 0. Therefore, the BER curve of COWDA is 
an upper bound for practical situations. 



'Or more generally when user data consists of {0, X}. 
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Fig. 5. Symbol Error Rate versus i] for different code signatures for the GCO (64, 88, {±1}, {0, ±1}) and WBE matrices. The 
GCO codes are not as fair as WBE codes but their performances are better than that of WBE codes. 



We have compared COWDA/ML decoder from the BER point of view with COW/ML decoder and 
BWBE/iterative decoder as depicted in Fig. [6l this figure shows the degradation of COWDA relative to 
COW/ML where there is no need for additional process of active user detection. As shown in Fig. [6l 
BWBE codes cannot reach error probabilities below a certain threshold even by increasing the power. 
Clearly, the BER of COWDA tends to zero as tj increases. This means that we can have a CDMA system 
with a desired low BER using COWDA matrices. For variable-active users, the sum capacity bounds will 
be discussed in Section IVIII 

IV. Sum Capacity Bounds for General User Inputs and Matrices- the Noiseless Case 

Let Z and S be two sets of (real or complex) numbers. In this section, we consider the noiseless 
CDMA channel Y = -^AX where the entries of the user input vector X and the signature matrix A 

v™ n 

belong to Z and S, respectively, i.e., X G A^nxi {^) and A G Mmxn {S)u 

To state our essential lemma about the expected mutual information, we should fix some notations: 

Definition 2 (The Probability of Difference of Two i.i.d. Random Variables): 



*For the noiseless case, the power constraint in ^ is irrelevant and therefore, r in ([5} can be set to 1. 
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Fig. 6. BER versus tj for a COWDA matrix with 64 chips and 88 users. The COW performance is for the case when all users 
are active. 



X, the difference set of X, is defined as: 

X = X-X={z-/|i,/ gX}. (36) 

For a probability law p{-) on X, we define p(-) to be the probability law on X which is the pdf of the 
difference of two independent random variables with the same probability distribution p{-). 

Finally, let 7r(-) be a probability distribution on S. The probability measure P'^ is the probability 
measure on M-mxni-S) induced by choosing entries of the random matrix independently and with the 
same distribution 7r(-). 



A. Sum Capacity Lower Bound 

In order to obtain lower bounds for the sum capacity for various scenarios, we prove a general theorem. 
Special cases are derived from this theorem. 
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Theorem 17 (A General Lower Bound for the Noiseless Case): 
In the absence of additive noise, for any Z and S, we have 

C(m,n,X,5) > supj-logE^ ('p('a^X = o)'"H , (37) 

where a £ S"' and X G X" with i.i.d. entries with distributions 7r(-) and p(-), respectively. 
The proof is given in Appendix 10 

For the case when the input and the signature matrix alphabets are finite, the above theorem can be 
simplified as follows: 

Corollary 4 (Finite Input/Finite Signature Matrix): Suppose that X = {ii, . . . , ig}. Let ki, . . . ,kq be 
nonnegative integers that satisfy ki + ■ ■ ■ + kg = n. Define B {ki, . . . , kg) to be the set of all n x 1 X 
vectors such that the number of occurrences oi ti, . . . ,ig in X are ki, . . . ,kq, respectively. It is easy to 
show that if Xi, X2 G S (fci, . . . , kg), then P (a^Xi = o) = P (a^X2 = o) =<l>{ki,..., kg, S). Thus, 
(|37] ) becomes 

Cim,n,I,S)> -log ^ (, "" ,)p{h)'''---p{^)'''^{h,...,kg,Sr. (38) 



B. Sum Capacity Upper Bound 

Below a general conjectured upper bound for the noiseless case is given. 

Theorem 18 (A General Upper Bound for the Noiseless Case): 
In the absence of additive noise, if X = {ii, . . . ,iq} with distribution p{ii) = Pi and S = {si, . . . , si}, 
we have the following conjectured upper bound: 



C{m,n,Z,S)< max <j min (n]H(X),mE[(/) ) [> , (39) 

p(-) 



in which 



where 5 is the Dirac function. 

Also, when Sj = e~i~v~^ and I divides n, we conjecture that ui = U2 = ■ ■ ■ = ui = j. 
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This conjecture is a special case of the conjectured upper bound for the noisy Theorem |23] when the 
noise variance is zero without restricting the signature to have unity magnitude. An upper bound that does 
not depend on the symbol alphabets but is simpler to evaluate is given in following theorem. However 
the bound is not as tight as the above upper bound. 

Theorem 19 (Noiseless Upper Bound): 

If |X| = g and |5| = I, then 

C(m,n,X,5) < m- max Vnf Mult (u,-, -,..., -| | , (41) 

where Mult (t, 6*1, ... ,6*^) denotes the multinomial distribution with probability (^ ^ ^^6^^ . . .6^" at 
(xi, . . . , Xq) when xi, . . . ,Xq are nonnegative integers and xi + ■ ■ ■ + Xq = t. 



Remark 2: It seems reasonable to believe that the maximum value in the above bound is attained at 
ui = ■ ■ ■ = ui = J and hence the bound may be simplified as 

C(m,n,X,cS) < mffl(Mult(y,-,...,- J J . (42) 

The proof is given in Appendix IF] 

C. Asymptotic Bounds 

Theorem 20 (Asymptotic Noiseless Lower Bound): 

Let p and vr be probability distributions on X and S with /i^ = and p be the distribution induced on 

X as before. Then 

1 f rank ( J ■ S) 1 

lim -C (m,n,X, 5) > min<^ ^ ^- -logp(J)} , (43) 

where J ■ S = {js\j ^ J,s £ S}, p{J) = Yl,ji^jP{J) ^i^^ for ^ ^^^ of numbers A, rank (A) denotes the 
dimension of A as a set of vectors over the field of rational numbers Q. 
For the proof see Appendix iGl 

Example 10: Let Z = S = {±1} and vr, p be uniform distributions on X and S. Therefore, X = 
{-2,0,2} and p{-2) = p{2) = \ and ^(0) = \. Now the maximum J C / with rank (J • 5) = 1 is 
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J = X with p{J) = 1 and therefore the bound in theorem becomes 





mm <; — - log ^, ^ - log (1) > = min I 1, — 



(44) 

This is the same result as the binary case given in (124] ) of Preliminaries. 

Example 11: Let I = {±1}, S = {±1, ±j} and vr, p be uniform distributions on X and S. Therefore 
X = {-2, 0, 2} and p (-2) = p (2) = i and p (0) = i. Now the maximum J C / with rank ( J • 5) = 2 
is J = X with p ( J) = 1 and therefore the bound in theorem becomes 

1 



mm<^-logi,A_log(l^ 



min < 1 



c 



(45) 



The above asymptotic bound and finite lower bounds for various values of m are depicted in Fig. [T] 
This figure shows that, unlike the asymptotic results for the noisy case that will be discussed in the next 
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Noiseless Asymptotic Lower Bound 

Lower Bound m — 32 

- ► - Lower Bound rii = 16 
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user: {±1} Signature: {±l,±j} 




Fig. 7. The asymptotic noiseless lower bound is compared to the normalized finite scaled CDMA systems. This figure shows 
that small to medium scaled systems cannot be accurately estimated by the asymptotic lower bound for high values of ^. 



section, medium scaled systems cannot be accurately estimated by the asymptotic lower bounds. 
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D. Examples 

Below, we will give many examples and simulation results that are special cases of Corollary |4l The 
derivation of the following formulas are all given in Appendix iHl 

Example 12: Binary Wireless CDMA, Z = S = {±1}, p{l) = pi, 7r(l) = vri 

C7(m,n,X,5)>sup{-log J] ( "^ Vp^ (1 -pi))'^+'^ (p? + (1 - Pif '' 



ki+k2+k3=n 



a=0 \ y \ ' 2 

When TTi = pi = 1/2, the above equation becomes identical to (l20l) in the preliminaries section; the 
simulation of (l46l l for the uniform distribution is given in Fig. [8] 

Example 13: Binary Optical CDMA, X = 5 = {0, 1}, p (0) = po, n{0) = ttq 

C(m,n,X,5) >sup{-log J] ( ^ V^^ (i _ ^3))^^+^^ (^^2 + (1 _ po)2)'^ (47) 

^"''"' ki+k2+k3=n ^ ^' ^' ^^ 

Fig. [8] also shows the above lower bound when po = ttq = 1/2. 



Example 14: Binary Complex Signature (Uniform Distribution), Z = {±1}, S = {l,j} 

/L"/2J . \ /I \' 

c(™,„,i,5)>-iogK:(^_^._;_g(-) 



L"/2J / „ \ / ^ \ n+2k / (2k\'^ "^^ 

\k . 
o2k 

k=o ----- — ' — ' • ^ 



(48) 



The above bound is also plotted in Fig. [H This figure shows that the lower bound for the complex 
signature matrix is slightly better than the optical case and the optical case is better than the binary 
wireless. We shall see in Subsection IVI-C.ll that under additive noisy environment, this statement may 
not be true. 
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Fig. 8. The sum capacity lower bound versus the number of users for binary input and binary signature matrices when all the 
probabilities are equal to 1/2 for m = 32. 



Example 15: Binary/Ternary, I = {±1}, S = {0, ±1}, vr (0) = ttq and vr (+1) = vr (—1) = vri 



cK„,x..),s.p{-i„.|:0(i)"(|: 



k 
a,a,k — 2a 



^k-2a2a 



(49) 



See Fig. |9] for the simulation result of the above equation. A slightly different bound for this system is 
given in [45]. 



Example 16: Binary/Ternary with Uniform Distribution, Z = {±1}, 5 is a set of ternary AIN's 



CK„,I.5)>-l„gt(,_,,:_,,)(i) 

fc=0 ^ ' ' / \ / 



32fc ^^ U, (3, 5 



(50) 
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The above bound is plotted in Fig. |9] 



Example 17: Binary/Ternary CDMA system (Uniform Distribution), Z = {±1}, S = {\,e^^ 3 } 

ri+ki+k2 



C(m, n,X, 5) > -log ^ 



n 



2,ki+k2 



E 



a+p+i=ki 



ki+k2+k3=n 

ki \ f k2 



^1,^2,^3/ V2 



a, /?, 6 J \a - !^,(3 - 1^,6 - ^ 



(51) 



The above bound is simulated in Fig. |9] This figure shows that the lower bound for the AIN ternary 
signature matrix is slightly better than the third roots of unity, which is significantly better than the 
{0, ±1} case. 
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Fig. 9. The sum capacity lower and upper bounds versus the number of users for binary input and ternary signature matrices. 
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Example 18: Ternary/Binary CDMA System, I = {0, ±1},5 = {±1}, p (0) = po, p(+l) = p(-l) = 
pi and 7r(+l) = ^ 

C(m,n,X,5)>sup{-log J] f, ," , ) (2p?)'^(4poPi)'^ (po' + 2p?)'^ 

<sC)(%.::-ja)"l^'- - 

The above bound is simulated in Fig. [10] and is compared to the ternary /ternary case of the next example. 



Example 19: Ternary Wireless System, I = S = {0, ±1}, p{0) = po, p{+l) = p(— 1) = Pi, vr(0) 
TTo and vr(+l) = vr(— 1) = vri 

Cim,n,I,S)> sup {-log J] f "^ V2p?)'^(4poPi)'=(pg + 2p?)'^ 



P^'^'''^') fci+fc2+fc3=n 



fc2 



^a+/3+<5=fcia'=0 ^ >A->/\,A-,z H/ j 

in which /?' = 2(a - /3) + a'. 

The above bound is simulated in Fig. [TOl This figure shows the upper and lower bounds for the 
ternary/ternary system, which is relatively tight. It also shows the lower bound of ternary/binary system 
of Example [TH which is lower than the ternary/ternary system. 

Example 20: Binary/Quaternary CDMA System when I = {±1} and 5 is a set of four AIN's 

-•''--•^)^--|(,..:-..)G)""(ii:(:)'(r)(":r))" <-> 

The above formula is simulated in Fig. [TT] and will be compared to the next example with complex 
signature but the same cardinality. 



Example 21: Binary/Quaternary System for I = {±1}, S = {±1, itj} 

,2k \ 22fc 



CKn,X,5)>n-logj;(;j(i|i) . (55) 

fe=0 
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Fig. 10. The sum capacity lower and upper bounds versus the number of users for ternary input and binary/ternary signature 
matrices. The lower bound for the ternary signature matrix is better than that of the binary case. 



Notice that the above formula is similar to ([7]). This implies that the lower bound for a binary/binary 
system with 2m chips is equivalent to the lower bound for binary/quaternary system with m chips. 
See Fig. [TT] for the simulation of the above formula. The lower bound for the signature with AIN 
quaternary matrix is much better than the dependent case. This is again due to the absence of noise; the 
differences are less pronounced in the case of additive noise. On the same figure, the lower and upper 
bounds of binary/pentads are plotted. This simulation implies that our lower bounds are relatively tight. 
In case of additive noise, the bounds are modified as shown in the following section: 



V. Sum Capacity lower Bound for General User Inputs and Matrices- the Noisy Case 

Under additive noise scenario, the evaluation of C {m, n, I, S, rf) is an extremely difficult problem but 
we will derive a family of lower bounds for C {m,n,Z,S,ri) for a class of random signature matrices. 
In the following general theorem, we let A = rB as in ([S]) for a fixed value of r and a randomly chosen 
B with distribution P'^. In this case the power constraint (HJ can be written in the following simpler form 
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Fig. 11. The sum capacity lower and upper bounds versus the number of users for binary input and quaternary/pentad signature 
matrices. 



after taking expectation over P'^: 



— {apnm{a^ + fi^) + fip n m{a^ + ^^)) < nrjaj 



or 



r < 



wj 



(cj2+n/i2)(cj2 +/z2)' 



(56) 



(57) 



A. Capacity Lower Bound 

The following theorem is the most general theorem for the sum capacity lower bound for any given 
input and signature matrix symbols with real/complex additive noise of arbitrary distribution: 
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Theorem 21 (General Lower Bound for the Sum Capacity): 



C {m,n,I,S,r]) > sup sup 

7r,p q 



-mE [q (iVi)) - logE^ ( ( E^^n, [ 2 '^"^^ ''^' 



(58) 



Where p(-) and 7r(-) are probability distributions on X and S, respectively, and q{-) is any arbitrary 
function. Also r satisfies the inequality given in (|57] ). and A'^i is the first entry of the i.i.d. complex noise 
N as defined in Subsection III-AI b and X are, respectively, vectors of length n with i.i.d. entries of 
distribution 7r(-) and p(-) as explained in Definition |2] in Section Hvl 
For the proof, please refer to Appendix Jl 

In the important special case of Gaussian noise. Theorem [2T] can be stated in a more explicit way by 
substituting q{x) = ^|f P lege in (1581 ): the result is given in the following corollary: 

Corollary 5 (General Lower Bound for the Sum Capacity for the Gaussian Noise): 
For a given Z and S, when noise is complex Gaussian with independent real and imaginary parts of 



variance 1 (hence a'j: = 2), dSSl ) becomes 



C{m,n,I,S,ri) > sup sup {-m (7 lege - log (1 + 7)) -logE^ ME;, f e^(i+^)-'''^^'' J J )}, (59) 



where r = , -r-, ttt-, — tt, b and X are, respectively, vectors of length n with i.i.d. entries of 

distributions 7r(-) and p(-). 



Remark 3: In the above corollary, the additive noise is taken as complex Gaussian with independent 
real and imaginary parts with zero means and equal variances. If the CDMA system was real, the 
imaginary part of the noise would not affect the capacity bounds. This implies that the bounds would 
have been the same if the noise was real with half the total complex variance. Having said this, we should 
note that since we have used the supremum over a family of Jensen's inequalities, the complex results 
are not analytically the same as the real one although the plots are very similar when we use numerical 
simulations. 

Remark 4: The general noiseless Theorem [17] can be derived from the general noisy Theorem |2T] or 
Corollary [5] when the additive noise power becomes negligible. 
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In the next 2 sections, we will derive and simulate special cases when the alphabet sizes are either 
finite {\T\ and \S\ < oo) or infinite (real/complex). 

VI. Results for Finite User Inputs and Arbitrary Signature Matrices- the Noisy Case 
In this section, we will discuss lower bounds, upper bounds, asymptotics, and simulation results. 

A. Capacity Lower Bounds 

In this subsection, we will develop bounds for {q, /)-ary and {q, Real /Complex) CDMA systems. 

I) Lower Bound for Finite User Inputs and Signature Matrices: 
For the finite user inputs and signature matrices, the general Theorem |2T] reduces to the following two 
corollaries: 

Corollary 6 (General Lower Bound for the Sum Capacity for (q, l)-ary CDMA Systems): 

Let X = {^l, . . . , tq}, X = {io,h, ■ ■ ■ , ^q}, h = ^ with distribution p and S = {si, . . . , si}. If p{ti) = pi 
and 7r(sj) = vtj, (|59l ) becomes 

C{m,n,I,S,r]) > sup {-m (7loge - log (1 + 7)) - log V] ( , ) TT^^ 



y' u .=k ^=l ^ ^'' ' " ' ^'^ t=l 

\ i<j<q 



m 



}, (60) 



where r 



J2_ 



{^l+ntil){al+iiiy 

For the proof see Appendix HI 

Corollary 7 (Results for Finite User Inputs and Symmetric Symbols for Signature Matrices): 
Let X = {?!,..., Zg}, X = {zo,zi, . . . ,?q}, ^0 = with distribution p{-) and S = {±si, . . . , zts/'j. 
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If p{ii) = Pi and 7r(sj) = 7r(— Sj) = ttj, ( [59l ) becomes 



C(m,n,X,5,7y)> _ sup {-m (7loge - log (1 + 7)) - log J] j.- r""''" 11 



9 



^ U«-'=oTTp'=. 



kn, k/i -'--'■•' 



E n 



fej 



Er=i"u+i'.i=fcj *~^ 



Uli,... ,Ul,i,Vu,--- ,Vl'i 
l<j<q 



J]^^ELi«*.+-tog,<^IELx-.{E|., iu,.-v..)W\' j 1^(61) 



t=i 



where r 



2<; 



Also when s;/ = 0, we have 

C{m,n,l,S,r])> sup {-m (7loge - log (1 + 7)) - log V (, "^ , V""^'" 17^? 



X 



Ki 



E n 

i<i<g 






Z'-l 
X 



Refer to Appendix |K] for the proof. 



2 j Lower Bound for Finite Input and Real/Complex CDMA Systems: 
In this subsection, we consider the case where the user data are finite and the signature matrix can have 
real or complex entries. 

Theorem 22 (Lower Bound for (q-ary, Real/Complex) CDMA Systems): 
For signature matrices with real or complex entries, let / = {iq, ±.ii, . . . , izg/}, where iq = , p(0) = po, 



34 



and p{+ti) = p{-ii) = Pi. 

C {in,n,I,W,r]) > sup {—m (7loge — log (1 + 7)) 

P(-).7 



-log Y. 

For signature matrices with complex entries, 



n \ ^.._^^ T-r .k, i , , '^lvT.]=okj\h 



~ |2 



^0, . . . , k^'j fi/^ I ^ m (1 + 7) (^2 + n/.2) 



}. (63) 



i=o 



C {m,n,I,C,r]) > sup{— m (7loge — log (1 + 7)) 

P(-).7 



7^ELo%l^i 



7 |2 



ELo '=.=" 



m (1 + 7) (cr2 + n//2) 



}• 



(64) 



See Appendix O for the proof. 

Special cases of Corollaries [6l |7J and Theorem |22] when the user inputs are either binary or ternary 
are given below: 



Example 22 (Lower Bound for the Sum Capacity for Binary/ l-ary System): 
For Z = {±1} and S = {si, . . . , si}, if p{\) = pi and 7r(sj) = ttj, (l60l ) becomes 



C{m,n,I,S,7]) > sup {— m,7loge — mlog (1 + 7) 



log J] 



fci+fc2+A:3=n. 



n 



ki,k2,ks 



ki+k2 2 



[piii-PiW^^'^ipi + ii-Pi) 



_ 11 ■=k-, t—i 






}, (65) 



where r 



2ri 



(<^?+«m?)K+mJ)' 



A special case of Corollary |7] when the user inputs are binary and the signature matrix is symmetric 
2r-ary is given below: 
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Example 23 (Lower Bound for the Sum Capacity for Binary/Symmetric 2l'-ary System): 
For I = {±1} and S = {±si, . . . , ±sp} and if 7r(+Si) = 7r(— Sj) = ttj and p(l) = i, (|6TI ) becomes 

n /n\ 

C{m,n,I,S,r]) > sup {-m (7loge - log (1 + 7)) -log^^ 






where r = \/ -^^ Also when sp = 0, the bound becomes 



C{m,n,I,S,r]) > sup {— m (7loge — log (1 + 7)) — log >^ ■ 



Tr{-),y 



k=0 



,fr. . V^o,...,«2(/'-i)/ M / 






A special case of Corollary |7] when the user inputs are ternary and the signature matrix has symmetric 
2/'-ary symbols is given below: 

Example 24 (Lower Bound for the Sum Capacity for Ternary/Symmetric 2l'-ary Systems): 
For Z = {0,±1} and S = {itsi, . . . , ±s//} and if 7r(+Sj) = 7r(— Sj) = vtj and p(0) = pQ^p{+\) = 
p(— 1) = pi , (|6TI ) becomes 

C{m,n,Z,S,r]) > sup {— m (7loge — log (1 + 7)) 

Pi,7r(-),7 



f E 



^1+^2+^3=1 

Ui,...,U2l'J \Vi,...,V2l' 

L 



E?!:i«.=fci 






(68) 
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where r = y z^-^- Also when si = 0, the bound becomes 



Pio- 



C{m,n,I,S,r]) > sup {— m (7loge — log (1 + 7)) 

pi,7r(-),7 



log E (, " ) 2''^+''W'^ {2p,p,f^ {2pl + pit 

E 



Uo,...,n2(p_i)y V^^o,---,^^2(/'-l) 



"0+Eiil Wi=fcl 

I \r^2(i'-i) , 

""0+1^1 = 1 Vi=k2 



I'-l 
X 



-TT ^«.+D,+«,'-i + .+t),'-i+,^^«o+i,Og^^YT^|2(n'=i'(2(«t-«i'-i+t)+^t-^i'-i+t)^t)l' ] \ (59-) 



A special case of Theorem |22] when the user inputs are binary and the signature matrix is real is given 
below: 

Example 25 (Lower Bound for the Sum Capacity for Binary/Real Systems): 
For Z = {±1} and 5 = M and if p{l) = \, (1631 ) becomes 

C(m,n,{±l},M,7?)>supf-m(7loge-log(l + 7))-log(X]^((l + ^)) jj- ^^0) 

B. Capacity Upper Bound 

Below a general conjectured upper bound is given: 

Theorem 23 (A General Upper Bound for the Noisy Case): 
If X = {ii, . . . ,iq} with distribution p{ii) = pi and 5 = {si, . . . , si} C S^ (unit circle), we conjecture 
that the capacity is achieved when the user distributions are i.i.d. We thus have the following conjectured 
upper bound: 

Cim,n,I,S)< max jmin Ue(X),m ( M(/) - log(— ) j U , (71) 

in which 

/»- E {ULZJ) (n»^-"-)/u-;^(E-E--)). <-. 

2_^j^iVij=Ui \k=l ^ / \fc=l / fc=l a=l 
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and 



fix + V^y) 



11 -riix^+y^) 



TT 



(73) 



Also, when Si = e ' ^ ^ and I divides n, we conjecture that ui = U2 
For the proof see Appendix |M] 



^1 = 1- 



C. Simulation Results 

1) Binary/Binary: A special case of Example |23] is simulated in FigUll The simulation is for (2, 2)-ary 
and when the spreading gain is m = 16. In the simulations, the input alphabets are fixed to ±1. Also 
simulation results show that when S = {l,e^^'^'"}, the lower bound is maximum. This simulation shows 
that the choice of symbols for the input and the signature matrix affects the sum capacity. The best lower 
bound is the signature with 2 AIN's. 
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Fig. 12. The normalized sum capacity upper and lower bounds versus the number of users for binary input and binary signature 
matrix when all the probabilities are equal to 1/2 for m = IQ and rj — 8 dB. 
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2) Binary/Ternary: The simulation results of Examples |22] and |23] are shown in Fig{T3] for the special 
case of (2, 3)-ary when the number of chips is equal to A, rj = 8 dB, and the binary input is equal to ±1. 
This figure shows a comparison of 3 different signature symbols; this simulation implies that the choice 
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Fig. 13. The normalized sum capacity upper and lower bounds versus the number of users for (2, 3)-ary when m = 4 and 
r; = 8 dB. 

of {0, ±1} for the signature matrix is not as good as far as the capacity is concerned. 

3) Ternary/Ternary: The simulation results of Examples [23] and l24l are shown in FigOfor the special 
cases of (3, 3)-ary, (2,3)-ary and (3,2)-ary. The input alphabet symbols are {0, ±1} for ternary and 
{±1} for binary users. The signature symbol is ternary {0, ±1}. As expected, the ternary/ternary system 
performs the best. For the quaternary matrix, the following example is instructive. 



4) Binary/Quaternary: The simulation results of Examples |23] and |24l are shown in FigiT5] for two 
special cases of (2,4)-ary systems. The user input alphabet symbols are binary {±1}. The signature 
symbols are either {ibl,ibj} or |1, 624,624 ,624 j that are AIN. This figure shows that the AIN case 
outperforms the "natural" quaternary signature matrix. 
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Fig. 14. The normalized sum capacity upper and lower bounds versus the number of users for ternary/ternary, binary/ternary 
and ternary/binary systems for m = 4, 77 = 12 dB. 



5) Binary/Arbitrary Matrix: When the user inputs are binary, we have simulated the bounds given in 
Examples [22ll25] for binary, ternary and complex quaternary signature matrices as depicted in Fig. [161 
the plots are for m = 16 and 77 = 20 dB. This figure shows that by increasing the cardinality of S, the 
lower bound is improved. This is a significant result since by adding extra complexity at the transmitter 
and receiver sides, we can increase the capacity. On the other hand, our asymptotic results as well as 
those of [12], [40] imply that the sum capacity is independent of the signature alphabet. 



D. Asymptotic Lower Bound for the Sum Capacity for Additive Gaussian Noise 

In this subsection, we consider the problem of estimating the capacity per user (c = -C) in the limit 
when the number of users n and the chip rate m go to infinity while the overloading factor /? = n/m 
and the normalized SNR r/ are kept constant. We first prove an asymptotic formula for the expression 
appearing in the right-hand-side of Corollary [5] 
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Fig. 15. The normalized sum capacity upper and lower bounds versus the number of users for binary/quaternary system for 
m = 4: and 77 = 8 dB. 



Theorem 24 (Asymptotic Lower Bound for Finite Input and Additive Gaussian Noise): 
For given sets S and Z (with \I\ < 00), and the corresponding probabiUty distributions 7r(-) and p(-) 
where Hp = 0, let b and X be vectors of length n with i.i.d. entries of distributions 7r(-) and p{-), 
respectively. Then 



lim 

m,n— ^00 


1 

n 


n/m^-13 





sup 

7 



inf 



-m^loge — logEv- 



1 




g2(l+7)7, 



1 + 7 




-(7loge-log(l + 7)) 

/5 



'2/3 V°H ^(1 + 7)^^(^U/^^) 



+ log 1 + 



2/3r/7A2 




(74) 



(1 + 7)^2(^2+^2) 

where Ai,A2 are eigenvalues of the covariance matrix of an rv. which has the distribution of the product 
of two independent variables with distribution p and vr, and B(-||-) is the KuUback-Leibler distance. 
Notice that for a binary CDMA system, the above lower bound becomes identical to that of (|25] ). 
The proof is given in Appendix JN] 
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Fig. 16. The normalized lower bounds for the sum capacity versus the number of users for various signature alphabets. In this 
figure X = {±1}, m = 16 and r? = 20 dB 



This theorem implies that the asymptotic bound does not depend on the symbol sizes or types of 
the user inputs or the signature matrix but rather on the probabilities of p{-) and 7r(-). This result is 
plotted in Fig[T7] This figure shows the normalized sum capacity bounds versus t] for binary, ternary, 
and quaternary inputs for /? = 1 and 3. Figure [18] is the simulation of several asymptotic results. This 
figure shows a comparison of the actual (soft decision) single user capacity for QPSK with the Guo- 
Verdu asymptotic result Q, and our asymptotic lower bound from Theorem [24] for I = {±l,±j}. For 
/? = 1, we can use orthogonal Walsh codes and hence the sum capacity is equivalent to the single 
user QPSK. Clearly, the asymptotic average mutual information derived by Guo-Verdu [12] for joint 
decoding QPSK is a lower bound for the actual sum capacity; however, for the hard decision QPSK, the 
Guo-Verdu bound is slightly better. For /3 = 3, the Guo-Verdu bound is between our lower and WBE 
upper bounds. This phenomenon is similar to that of the Tanaka's and our bounds for the binary case [15]. 



In the next section, the previous theorems and corollaries are extended to the case when not all users 
are active. 



42 




,Q,..,43-.-.e--'-G-.— e---e--.©--©- 



Binary 



: /3 = 1 input {±l,±i} 

/3 = 3 input {±l,±i} 

■ - - /i = 1 input < 1 , e^ "3- 
/3 = 3 input |l,e±T 

O i^ = 1 input {±1} 
/i = 3 input {±1} 



20 

'; (dB) 



25 



30 



35 



40 



Fig. 17. The asymptotic lower bounds for the normalized sum capacity versus i] for binary, ternary, and quaternary inputs for 
P — I and 3. The matrix is irrelevant as far as the asymptotic results are concerned. 



VII. Capacity Bounds for Variable-Active Users 

In Subsection IIII-EI we defined active users and overloaded codes for the binary case that were errorless 
(COWDA) with active user detection capability. In this section, we would like to find the sum capacity 
bounds for the variable-active users for AWGN channels. We consider two main scenarios: 1- A;-active, 
and 2- pact-^ctive. The term "fc-active users" implies that only k out of n users are active at any given 
time for a CDMA system (signature time length). The term p^ct-active implies that each user is active 
with the probability pact- This concept may also have applications to compressive sensing of sparse 
signals [46]; the signature m x n matrix becomes the sampling m x n matrix and the original signal 
is /c-sparse. In the compressive sensing applications, typically, 2k < m < n; this is is analogous to an 
overloaded CDMA system when only /c-users are active. The following two subsections extend the result 
of Theorem |22] to the k- and pact-^ctiwe scenarios. 



A. Capacity Lower Bounds for a k-Active CDMA System 

In this subsection, we will consider 3 scenarios: 1- Binary /Real, 2- Real/Real, and Binary /Binary 
CDMA systems 

1) Capacity Lower Bound for k-Active Binary/Real CDMA Systems: 
The following theorem extends the result of Theorem |22] to the A;-active scenario: 
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Fig. 18. The asymptotic actual, upper and lower bounds for the normalized sum capacity versus r\ for QPSK inputs for /3 = 1 
and 3. In this figure, our lower bound is compared to that of Guo-Verdu and the single user sum capacity. 



Theorem 25 {Capacity Lower Bound for k- Active CDMA Systems, X={ibl}, S = M): 
If X is a A;-active binary input vector with i.i.d. entries and the distribution on S is A/'(0, 1), then 



Cactive(?7i,n,/c,{±l},M,7?) > sup{-m(7 log 6 - log (1 + 7)) 

7 



log 



1 



«J 



n 



- S^ 2^~^^ ( 1(1 + 

'^''ilf ^ Vi'^+i-«>^-2j>- V V fcm(7 + l^ 



Airi'yr] 



}■ 



(75) 



The proof is given in Appendix lO.il The above lower bound is also simulated in Fig. [19] for r/ = 20dB, 
and will be discussed at the end of this subsection. 



Below, we will discuss active users for real inputs and signature matrices: 
2) Capacity Lower Bound for k-Active Real/Real CDMA Systems: 



For the fc-active user case when the probability distributions on X = 5 = R are M (0, 1), we can find 
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Fig. 19. The sum capacity lower bounds w.r.t. activity factor for various scenarios when ri — 20dB. 



the lower bound as shown below: 



Theorem 26 (Lower Bound for k-Active Real/Real CDMA Systems): 



Cactive(m-,"-, ^,]K,K,??) > sup{-m(7loge-log(l + 7)) -log ^ V'O')}, 

jf=max(0,2fc— n) 



where 



V' (j) = < 



1 r°^ 



2T{k) 



r 1+ 



2'yxnri 
(l+7)fc'm 



X e 2 dx 



j = 



(T^^^^TT^riiiMiy/o^^/on^ + tJ^) ' -^--V-e^e^d... ,> 0.(76) 



See Appendix 10.21 for the proof. 



The above lower bound is also simulated in Fig. [19] for rj = 20dB ; the explanation of this figure will 
be discussed in the next subsection. For the special case of real user inputs when all users are active 
(k = n), we have the actual sum capacity and therefore the lower sum capacity can be compared to the 
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Fig. 20. The sum capacity lower bound is compared to the actual sum capacity for real valued inputs (r\ = 20dB). This plot 
highlights that our sum capacity lower bound tracks the average capacity. 



actual one. Simulation results depicted in Fig. |20] imply that the sum capacity lower bound is relatively 
tight and closely tracks the actual capacity. Also, on the same figure, the average capacity derived from 
© is plotted. This average is derived from randomly generating real signature matrices and applying © 
in the Preliminaries. Our lower bound is derived from this average value. The capacity lower bound for 
a binary/binary A:-Active CDMA System is given below: 

3) Capacity Lower Bound for k-Active Binary/Binary CDMA Systems: 
When the binary users are inactive, the input is equivalent to a ternary {0, ±1} system, where the 
probability of is equal to k/n. The lower bound is derived from the following theorem: 



Theorem 27 (Capacity Lower Bound for k- Active CDMA Systems, X = S 
If X is a /c-active vector with i.i.d. entries with binary distribution, then 



{±1}|- 



Cactive{'m,n,k,{±l} ,{±1} ,vi) > sup{-m(7loge-log(l + 7)) - log (;u(/c, 77,7))}, (77) 

7 
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where 




J 



. j+, „ ,e™Mi+.) . (78) 

For the proof, refer to Appendix 10. 3 1 The above lower bounds are simulated in Fig. [19| with real values. 
This figure compares three scenarios, namely, real/real, binary/real and binary/binary for two different 
values of m and n. For a large scaled system (m = 32 and n = 64), the binary/real and binary/binary 
CDMA systems coincide while for a small scaled system (m = 4, n = 8), the binary/binary system is 
somewhat less. These plots confirm that for large scaled systems, the lower bounds become independent 
of the signature alphabets. 



B. Capacity Lower Bound When the Probability of Active Users is Known 

When the users are active with the probability pact, the capacity lower bound can again be derived 
from Corollary [5] For large scale systems, the A;-active CDMA case discussed in the previous subsections 
become equivalent to the case when pact = k/n. In the following subsections, we will discuss various 
scenarios for the user inputs and signature matrices: 

1) Capacity Lower Bound for a Real/Real pact-Active CDMA System: 
In this subsection, we shall derive the sum capacity lower bound for the p^ct-active real inputs assuming 
the matrix is also real. We will also derive the asymptotic case for the large scale system. 

Theorem 28 (Active Users with Probability pact, X = S = Mj.' 
For a CDMA system with real input, let pact be the probability that the users are active, then we have 



Cp^^,(m,n,R,M,Ty,pact) > sup{-m (7loge - log (1 + 7)) 
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where 
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See Appendix 10.41 for the proof. The above bound is simulated in Fig. |2T] and will be discussed at the 
end of this section. 

2) Asymptotic Lower Bound for a Real/Real pact-Active CDMA System: 
The asymptotic sum capacity for the above equations is given below: 

Theorem 29: Asymptotic Lower Bound for a Real/Real pact-Active CDMA System 



n,m™oo -Cp..Am,n,R,R,r],p^ct) > --^ (7loge - log (1 + 7)) - sup{(F (x) - / (x)}, 



a;>0 



where 



and 



F (x) = -^ log (1 + -|^x 

2/3 V (1 + 7) Pact 



I (x) = supllog e X xt - log ((1 - Pact? + ^"""^^f _ ^^"'^ + ;7=| ) } 
For the proof see Appendix 10.51 



(81) 



(82) 



(83) 



3) Capacity Lower Bound for a Binary/Real Pact'-^ctive CDMA System: 
When the inputs are binary (bipolar) and the signature matrix is real, we have the following theorem for 
the paci-active CDMA System: 



Theorem 30 (Binary Input with Real Signature): 
Cp„,,(m,n,{±l},M,7?,pact)>sup{-m(7loge-log(l + 7))-log J] 2^'""^' (t T t ) 

27??(4A:i + k2 
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t''=^+'Hl-Pact)'Ml + T^Pa'et-2pact 
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mpactil + 1) 



}.(84) 
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This theorem can be derived from Theorem l22l when X = {0, ±1}. The simulation of the above bound is 
shown in Fig. [21] and will be discussed in the next subsection. The asymptotic formula for the pact binary 
input case is the same as the general asymptotic derivation given in (1741 ) when the input is ternary and 
the probability of is 1 — pact- The simulations of the above formula along with the asymptotic bound 
derived from (1741 ) are depicted in Fig. |22l This figure shows that for even small values of ttt, = 16, the 
asymptotic bounds and the finite bounds derived from the above evaluation for /3 = 1, 2, 3 coincide. 
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Fig. 21. The normalized sum capacity lower bound for pact-active CDMA when m = 32, n = 64 and r\ — 20dB. This figure 
shows that for large scale systems, the bound is independent of the signature matrix. 



4) Capacity Lower Bound for a Binary/Binary pact-^ctive CDMA System: 
When the inputs and the signature matrix are binary (bipolar), we have the following theorem for the 
Parf-active CDMA system: 
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0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 



Activity Factor {pa 



Fig. 22. Tlie normalized asymptotic sum capacity lower bounds w.rt. pact as compared to finite cases for binary input. Here 
rj — 20dB, m — 16, P = 1, 2, 3, and the signature matrix is arbitrary. 



Theorem 31 (Binary Input and Binary Signature): 

Cp^at {'m,n,{±l},{±l}, r], pact) > sup{-m(7 log e - log (1 + 7)) 



-log Yl 2^'""' 

ki+k2+k3=n 



n 






V'-iW 



kl\ fko\ -2ri-,l4i + 2j-2ki-k2)^ 

I / \ 3 • 



(85) 



For the proof please refer to Appendix 10.61 Fig. |2T] is the simulation of the sum capacity for the Pact- 
active CDMA case for the above examples. The simulations are for m = 32, n = 6A and t] = 20dB. This 
figure shows that for large scale systems, the bound is independent of the signature matrix. The lower 
bounds have a maximum for pact ~ 0.65 because of the extra information needed for the determination 
of active users which is approximately equal to log (^ " ) . 
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VIII. Summary of the Main Results, Conclusion, and Future Work 

In this paper, we have attempted to reveal some of the unknowns regarding CDMA. Our first concern 
was related to the developments of overloaded errorless codes (referred to here as GCO code) for general, 
finite/infinite, and real/complex users and signature matrices. A general theorem (Theorem [141) was 
developed to construct larger GCO matrices from the smaller ones. Various examples were given for 
special binary/ternary user inputs and signature matrices. The same procedure was extended to complex 
signatures. A general upper bound for the overloading factor (n/m) was developed and simulated for a 
special case. From these theorems, practical ML algorithms were suggested to decode large size GCO 
codes. Simulation results for special cases showed that GCO matrices outperformed the WBE matrices. 
The fairness of COW/GCO codes were simulated and discussed; GCO codes were not as fair as the WBE 
codes but even the worst GCO code was better than the best WBE code for moderate SNR values. If 
some of the users are not active, the special class of GCO codes (named COWDA) were simulated and 
compared to WBE and COW codes. COWDA matrices lack the same performance as the COW codes 
where there is no need for active user detection but they outperform the WBE codes. 

Our second concern was on the evaluation of the bounds for the sum capacity for arbitrary input 
symbols and signature matrices. The next 4 sections were devoted to this problem. Section |IV] is related 
to the noiseless case. In this section, a general theorem (Theorem [T71 ) was developed to find a lower 
bound for the sum capacity. From this theorem, a corollary for finite input/matrix and examples for 
the special cases were derived and simulated. The simulations show that there is a linear region, where 
the lower bound is very tight up to a point at which the linearity suddenly breaks down in a severely 
loaded region (nth)- We conjecture that the maximum number of GCO users is nthim) for any given 
m. Also, the simulations showed that by choosing proper symbols for the signature matrix (such as 
algebraically independent numbers) and a probability law for the symbols, the lower bound could be 
drastically improved. The simulations also showed that by choosing matrix entries from larger sets (|X|), 
we could increase the linear region of the lower bound and hence nth. 

In Section lYl we developed a general theorem (Theorem |2T]) for the sum capacity lower bound for an 
arbitrary CDMA system for any types of symbols, size, probability distributions, and additive noise. The 
general noiseless Theorem [TT] can be derived from the general noisy Theorem [2T] when the additive noise 
power becomes negligible. The important special case of additive Gaussian noise is given in Corollary 

E 
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Section |Vl] is the special case of finite symbols for the user inputs and signature matrices. Corollaries 
[22H24l were special cases for binary /ternary user inputs and /-ary signature matrices. Many examples and 
simulations were given in this section. Again, the dependence of the lower bound on the symbols, sizes, 
and the noise level is apparent. In this section, an asymptotic lower bound was derived, which implies 
that for large scale systems the bound does not depend on the signature alphabets but rather on their 
inputs. Subsection IVI-A.2I is related to the development of the sum capacity lower bound for binary 
users but real or complex signature matrices. The simulations show that by changing the signature matrix 
from a finite set to an infinite real/complex number, the sum capacity can be improved significantly. This 
is a noteworthy result since by adding extra complexity at the transmitter and receiver sides, we can 
increase the capacity. However, our asymptotic results, as well as those of [12], [40], imply that the sum 
capacity is independent of the signature alphabets. Nevertheless, our simulations for finite n and m show 
significant differences for the sum capacity lower bounds for different signature symbols, probabilities, 
and cardinalities. 

The sum capacity lower bound was extended to the /c-active case in Section IVIIl which may have 
applications to compressed sensing. In this section, we developed the sum capacity lower bound for 
the special cases of binary /real, real/real, and binary /binary /c-active users; the lower bounds were also 
simulated and compared with respect to the A;-activity. For the special case of real user inputs when all 
the users are active, we have the actual sum capacity and therefore the sum capacity lower bound was 
compared to the actual one. Simulation results showed that the sum capacity lower bound was relatively 
tight and closely tracked the actual capacity. We also developed similar results for pact-activity with 
simulations; in addition, an asymptotic lower bound for the real/real case was derived. 

As for future work, we suggest to study the effects of fading due to multipath on injectivity of GCO 
matrices and the evaluation of the sum capacity bounds. The problem of asynchronous CDMA would 
be a productive area of investigation as indicated by several attempts already made [30], [47]-[49]. The 
extension of our results to near-far effects is another challenging task [50]. Also, the consideration of 
generalized users [51]-[54] is yet another interesting topic to look rntcL. 



'a generalized user chooses its data from a set of variable vectors with different symbols. 
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Appendix 

A. Proof of Theorem [74] A General Method for Constructing GCO Matrices 

The necessary and sufficient condition for B to be one-to-one is Ker{B} p| {—5-} = {0}". Now if 
Z G Ker{5} fl {^}". we claim that Z = 0. We know that 



BZ = 0, where Z 



yT yT 



(86) 



and each Z,; is an n x 1 vector for i = 1, 2, , . . . , u), Z^+i is an m x 1 vector and the entries of Z, are 
from X — X for i = 1, . . . , u; + 1. Thus ( l86l ) implies that: 

m^ [CZi, . . . , CZ^]^ = - [MiZ^+i, . . . , M^Z^+i]^ . (87) 

Multiplying by H^, we get 

kw\^ [CZi, . . . , CZj^ = -H^[MiZ^+i, . . . , M^Z^+if. 

We assume that the first column of H^; does not have any —1. Thus, we obtain 

w 

kw{CZi) = -^MiZ^+i. 
Hence 



i=l 



CZi 



i^(l'^)^-« 



kw 



Vo 



\ 



-t 

kw , 



z 



u)+l- 



The left hand side of ( |90b is an integer vector and because 



uwk „1 X — X 

■.u^%\ n^-— = {0}, 



all entries of Z^+i are 0, therefore 



Hence 



kvAw [CZi, . . . , CZ^„] = 0. 



(CZi)=0 i = 1,2, ...,u;. 



(88) 



(89) 



(90) 



(91) 



(92) 



(93) 



Since C is a GCO (m, n,X, 5) matrix, Zj = for i = 1, 2, . . . , w, which implies Z = 0. This completes 
the first part. Now let S consists only of odd numbers, then Z^+i is a (m — 1) x 1 vector and (|90l ) 
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becomes: 



kw 



-1 



^^1 = 7:;:; E^O^^+i 



kw 



a=l 



~t 

kw 



z 



w+1- 



(94) 



. 
\0 ... 0/ 

The left hand side of ( |94l ) is an integer vector with the same parity as its entries and because the last 
entry of the right hand side vector in (l94l ) is 0, all the entries should be even and again because 



(2uwk ^ X-X 



w, 



all the entries of Z^^+i aie 0, and the rest of the proof is similar to ( l92l ) and ( |93] ). 



(95) 

D 



B. Proof of Corollary [7] Constructing Binary GCO Matrices 

S consists only of odd numbers ±1. We set /c = 1 and w = 4r in Theorem [141 note that Mj's are 
binary matrices such that X]i=i ^i — ^^m- Thus, t = 2 and it suffices to show that 

{Aru :ueZ}r\{X-X) = {0}. (96) 

The above equation is true because the minimum nonzero positive entry of {Aru : u G Z} is 4r and the 
maximum positive entry of T — X is 2q, and by our hypothesis, 4r > 2q + 1. D 



C Proof of Theorem [75l Generation of Overloaded Complex GCO Matrices for Arbitrary Input Integers 

The necessary and sufficient condition for B to be one-to-one is Ker {B} P| (X — X)" = {0}". Now 
if Z G Ker {B} ^ {X - X)", we claim that Z = 0. Suppose that 

, T 

(97) 



^[Zo Zi ...Zu Yi Y2 ... Yk) =0, 

where Zi and Yi are, respectively, 1 x n and 1 x (m — 1) vectors. Since {1, ^i, . . . , ^^j are independent 
numbers, the coefficient vector that is multiplied by fj is the zero vector for i = 1, 2, . . . , fc. Thus, we 
have 



^iiUiZi^) + ^iM'Y^ = for i = l,2,...,fe. 



(98) 



Hence 



(D, M*) (ZiYi)^ = for i = l,2,...,k. (99) 

because [Dj M*] is a GCO (m, n,X, {0, ±1}), We must have (ZiYi)'^ = for i = 1,2, . . . ,k; because 
Do is injective on X, Zq = 0, and thus Z = 0. 
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D. Proof of Theorem [76] A General Upper Bound for GCO (m, n, X, S) 

Let the input multiuser data be defined by the random vector X = [xi, . . . ,x„] , where Xj's are 
i.i.d. random variables from the set Z with uniform distribution. Since Xj's are independent, we have 
H {X) = nlog2 |X|. Now, let the transmitted CDMA random vector be defined hy Y = AX, where A 
is a GCO (m, n,X, 5). Since A is one-to-one over X", we get M.{Y) = EI(X) = nlog2 |X|. But we have 
^ {y) < TZi M {y^) < mmax^a,,...,a^)eS'^ H ivi), where yi = Z]=i ajXj. D 

E. Proof of Theorem \T7\ A General Lower Bound for the Noiseless Case 
For a given A, 



1{X; Y) = m{Y) = -Y^^{Y = y) log P(y = y) 
y 

= - J] P(X = x) log P(y = Ax) 



> -log 



J]Px(x)Px(x)l{A(x-x')=0} 



r^Pxwiog 


I 


X 


Ix 


0=0} 


= -log 






L 



J^Px(x')l{Ax=Ax- 



'} 



E^xWl{An= 



:0} 



(100) 



where the inequality is due to Jensen. Now taking expectation with respect to A and using Jensen 
inequality once more, we have 



E^(I(X;y)) >Ea f-logJ]P^(n)l|A„=o}) 
= -logf J]P^(tx)P(An = 0)J. 



But since the rows of A are i.i.d., (An)^^, . . . , (Au),^ are likewise i.i.d., and thus we get 

P (An = 0) = P ((An)i = 0, . . . , (An)„ = 0) = P ((Au)^ = 0)" = P {a.u = 0)' 
Therefore, E^ {^X-Y)) > -log (Eu^xW^la--" = 0)"") = -logE^ ((p(a.X = 0) 



(101) 

(102) 

D 
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F. Proof of Theorem [79] Noiseless Upper Bound 

Let Z = {ii, . . . ,iq] and S = {si, . . . , s;}. Clearly, the capacity is maximum when the numbers liSj 
are AIN's; for this case we have: 

m 

l{X-X)=n{Y)<S^n{Yi)<m- max H(Yfc)- (103) 

^ — ' l<k<m 

i=l - - 

But Yk = YTj=i ^kjXj and if the number of Sj's in the vector [a^i, . . . , akn] is Ui, then Y^ = sizi + 
••• + sizi where Zi has the distribution: Mult {ui,p (ii) ,p (12) ,■■■ ,p{iq)) and Sj's are independent. 
The algebraically independence condition implies that the map {zi, ..., zi) ^ Y^ is 1 — 1 and therefore 
H (Yk) = X]i=i ^HI (^j)- On the other hand, according to a theorem from Shepp and OMn [55], the entropy 
of a multinomial distribution Mult {t,6i, . . . ,6t) is concave and symmetric with respect to 6*1, . . . , ^i and 
hence attains the maximum at 9i = ■ ■ ■ = 6t = j. D 

G. Proof of Theorem [20] Asymptotic Noiseless lower Bound 
To prove, we need the following lemma: 

Lemma 3: Let S = {■?!, • • • ,C(i} be a finite set of numbers and a a non-degenerate probability 
distribution on S with jj-a- = 0. 
If Xi,X2, . . . are i.i.d. with distribution a then 

F{Xi + X2 + --- + Xn = 0) = O(n —). (104) 



Proof of Lemma \3} 

Let ki = #{je{l,..., n} \Xj = ^i} and F = {{a, ...,Cd)£ Z'^|ci6 + • • • + Cd^d = O}. Clearly F 
is a (d — rank (S)) -dimensional lattice in Z^ and Xi + ■ ■ ■ + X^ = iff A; = (fci, . . . , A;^) G F. 
But kj ~ B{n,aj) and hence k'- = '-' ~J^°^"' ^ 7V(0,aj (1 — iTj)). But since /Xo- = thus if fc G T, 

A;'i6 H ^ ^dCd = and so we have k e F Hi k' = {k[, . . . , k'^) e -j^{F - y/na). 

Now k' has an asymptotically fixed non-degenerate d-dimensional Gaussian distribution and F is a lattice 
of co-dimension rank (S) and hence -j= {F — y/na) is of O ( i^^) ) measure. D 

Proof of Theorem: 
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We know that 



-C{m,n,I,S)>-- log E^ (f (a.X = o) ™) . (105) 

Now let p be a probability distribution on I and let X = £ where the empirical distribution of x is p. 
Moreover, assume that p is supported on J C X. For large values of n, a.x may be considered as sum 
of n i.i.d. r.v.'s with the non-degenerate distribution p (g) vr from J ■ S and hence according to Lemma [3j 

p (a.X = 0) = O (n ^ J . (106) 

Note that this probability might dependent on p and n so we rewrite (11061 ) as: 

F{a.x = 0)<f{p,7r)n~'-^'^^ (107) 

and therefore: 

F{a.x = or < 2-'°g/tf.-)- '"'°-"r"^' = 2-"("=^-f i°s^M = 2-"("=^+°(i)). (108) 

But recall that the probability of appearance of p on the empirical distribution of X is approximately 

2-nDip\\p) and therefore: 

lim -C(m,n,I,S) > max lim -i log f2-"^(*).2~"(=^^+^(i)) 

m log II ^ ponj 

min|zj(p||p) + '"'"^^f '^H - (109) 

ponj 

But it is easy to see that minp supported on j D {p\\p) = — logp (J) which is attained where p is proportional 
to p on J and this finishes the proof. D 

H. Proof of Examples of Corollary |4] 

1) Proof of Example \12\ Binary Wireless CDMA: 
Note that X = {+2, -2,0} with probabilities p(+2) = p{-2) = pi{l-pi), and p{0) = pi^ + (1 -pi)^. 
Let xi G B{ki, /c2, k^)', suppose that a^.xi = and a^ has a "+1" entries and /3 "+1" entries in places in 
where xi is equal to +2 and —2, respectively. We conclude that 2(a — (fei — a) — /3 + (/c2 — /3)) = or /3 = 
a+{k2-ki)/2. The number of such a"^ is equal to (^') (^') with the probability 7ri"+^(l-7ri)'''i+'''2-°~^; 
thus e{ki,k2, fes, 5) = Y.''' a=o (a) Cfl^i^^^C^ " ^1)''^+''^"""'^ and this completes the proof. 
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If vTi = pi = 1/2, then due to symmetry it is easy to show that if the number of nonzero entries 
in X is a fix number like k then the value of p{a^ .x = 0) remains constant, i.e. ±2 behave like 
a single symbol and X = {±2,0}. Thus if xi G B{k,ki} and a^.x = 0, then k must be even 
and p{a^ .X = 0) is equal to (k)/2''. Hence if q = p = 1/2, the bound reduces to the bound in 

2 

[14]. D 

2) Proof of Example \13\ Binary Optical CDMA: 

The proof is similar to the proof of example [121 D 

3) Proof of Example \14\ Binary Complex Signature (Uniform Distribution) : 

Note that Z = {+2,-2,0} with probabilities p^^ = p^^ = \,pi,^ = \. Let xi € B{ki,k2,kz); suppose 
that a^ .xi = and a^ has a "+1" and j3 "+1" entries in positions where xi is equal to +2 and —2, 
respectively. We conclude that 2(/ci — a — A;2 + /3)j + 2(a — /3) = 0; thus a = f5,ki = k2 — k . The number 
of such a^'s is X]a=o (a) (a) bccausc we should choose a entries from those +2 entries in xi and a 
entries from those —2 entries and let the corresponding entries in a^ be 1 and let the remaining 2k — a 
entries be j. Each a^ has probability (i) and since X]q=o (o) (a) ~ Ck)' ^^ S^^ ^(^' k,n — 2k, S) = 

afELoOO=afCk')- □ 

4) Proof of Example [73] Binary/Ternary, I = {±1}, S = {0, ±1}, vr (0) = ttq and vr (+1) = vr (—1).' 

If p(+l) = p(— 1) = ^, it is easy to show that if the number of nonzero entries in x is equal to 
k, then the value of p{a^ .x = 0) remains constant. In other words ±2 behave like a single symbol 
and I = {±2,0}. Now, suppose that xi E B{k,ki} and the nonzero entries of xi are all ±2 and 
a^ .xi = . If a"^ has a "±1" and /3 "—1" entries, in positions where xi is equal to 2, we conclude 
that a = /3 . The number of such a'^ is (at) ( a°) each with probability vr^^^^vrf". Thus e{k,ki,S) = 

^^^fc f k\ f2a\k-2a^2a\ r-i 

l^a=o[2j[a)^0 ^1 ) ■ ^ 

5) Proof of Example \T6\ Binary/Ternary Uniform distribution: 
The proof is similar to proof of example [17] 

6) Proof of Example \17\ Binary/Ternary Uniform Distribution: 

Note that X = {+2,-2,0} with probabilities pi^ = pi^ = \,pi3 = ^; Let xi G B{ki,k2,kz); suppose 
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that a^.xi = ciw"^ + C2W + C3 = 0. Because w is a the third root of unity, one can deduce that 
ci = C2 = C3 = c. If we denote s as the sum of entries of xi, we have ci + C2 + C3 = s = 
2{ki — k2), which yields c = ^^ " . In order to satisfy a^.xi = 0, one should choose ai entries 
such that xi is equal to +2, and ai — § entries such that xi is equal to —2. Let the correspond- 
ing entries in a^ be 'uP' and the same argument for w and 1 entries of a^ . Thus the number of 
a^'s which satisfy a^.xi = is equal to {y. t\){^'Z'){^jA^)tl'^^t^)\ with probability 

7) Proof of Example [78] Ternary /Binary System, X = {0, ±1},5 = {±1}, p (0) = po> p(+l) = 
p( — 1) = pi ant/ vr(+l) = i; 

Because vr(+l) = vr(— 1) = ^ and p(+l) = p(— 1), it is easy to show that if the number of ±2 
and ±1 entries in x are fixed, then the value of p{a^.x = 0) remains constant. In other words, ±2 
and ±1 behave like +2 and +1 symbols. Let Z = {^l,^2l*3} = {=t2,ibl,0} with the probabilities: 
Pjj = 2pi^,pj2 = 4pipo,pi,j = pq^ + 2pi^. Let xi G B{ki,k2,ks) and a^.xi = 0; from the above 
argument we can assume that the nonzero entries of xi are all +2 or +1. If a^ has a "+1" entries where 
2:1 = +2 and /? "+1" entries where xi = +1, we get a^ .x = 2{2a — ki) + 2(3 — k2 = 0, which yields 
P = ^ + ki-2a. The number of such a^ , is (^') (^^') with probability (^)^^+^^ By applying Corollary 
H we get ( |52b . 

Sj Proof of Exampleim Ternary Wireless System, I = S = {0, ±1}, p (0) = po. p(+l) = p(~l) = Pi> 
7r(0) = ttq and '7r(+l) = vr(— 1) = tti: 

By the same argument in the proof of ExamplefTSl let X = {^i, 12,^3} = {i2,=bl,0} with the probabilities 
Pjj = 2pi^,pj2 = Aplpo and pi^ = po^ + 2pi^. Let xi € B{ki,k2,kz) and a^.xi = 0; we can assume 
that the nonzero entries of xi are all +2 or +1. If a^ has a "+1" and /3 "—1" entries when xi = +2, 
respectively, and a' "+1" and /3' "—1" entries when 5;i = +1, we get a^.x = 2(a — /?) + a' — /3' = 0, 
hence /?' = 2(a - /?) + a'. The number of such a^'s is (^j) C"^^") (^^,) C"''^"') with the probability 
^^Q+/3+a'+/3'7^^fci+fc2-a-/3-a'-/3' gy applying CorollaryHwe get ^. 



9) Proof of Example \20\ Binary/Quaternary System for I = {±1}, S is a set offourAIN's: 
Let 5 = {si, S2, S3, S4} and note that I = {+2,— 2,0} with the probabiUties pi^ =^2 = 1 ^rid pi^ = ^. 
Let xi G B{ki,k2,k3) and assume that a^ .xi = cisi + C2S2 + C3S3 + C4S4 = 0. Because Sj's are 
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independent numbers, one has ci = C2 = C3 = C4 = 0; thus = ci + C2 + C3 + C4 = 2{ki — ^2) which 
yields ki = k2 = k. Note that to satisfy a^ .xi = 0, one should choose ai entries when xi = +2, and ai 
entries when xi = —2, and let the corresponding entries in a^ be si and the same argument for 52,53 
and S4. Thus the number of a^'s which satisfy a^.xi = is equal to ^ai,a2,«3,«4 ( ^ Iq, ta 'a ' ) ^^'•^ 

-. 2A: 

probability (|) . The following identities yield (1541 ): 



E ' " 



01,02,^3,04 \ j^ ^ •' '* 



01,02 ^"1 '"2 7 (a3 + a4)!^„t^, V a3!a4! / ^^^ V^iiaaV (a3 + a4)!^V "3 + 04 



a,^,m Ui!«2!y ^\m ) ~ ■^^[m ) {k-m)\^ ^^[ ai!a2! 

Oi+02+m=A; 

_A 1 /2m\ A:!^ /2/c - 2m\ _ A / A; \ Y2m\ /2A: - 2m\ 



D 



10) Proof of Example 127] Binary/Quaternary System for T = {±1}, 5 = {±1, itz} ; 
If p(+l) = p(— 1) = \ and vr is a uniform distribution, it is easy to show that if the number of nonzero en- 
tries in X is equal to k, the value of p(a-^.x = 0) remains constant; i.e ±2 behave like the single symbol +2 
and i = {±2, 0}. Now assume that xi G B{k\ ki} and the nonzero entries of xi are all +2 and a^.xi = 
. If a^ has a "+1", /3 "—1", a' "j", and /3' "— j" entries in positions where xi is equal to 2, we get 
2(a— /3)+2(a'— /3')j = 0; thus, a = /3, a' = /?', so k' = 2k is an even number. The number of such a^'s is 
Eto (2-)(?)ftr) with probability (i)^^ Hence, ^(^',^^,5) = (eIo (2-) ( 'O ftf) (I)'')- The 
combinatorial identities yield (eIo O (?) CtT) ilf) = (P) '■ □ 
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/. Proof of Theorem |27] General Lower Bound for the Sum Capacity 
For a fixed A, we have 

1{X;Y) = h{Y) - h{N) = Ey(-log/y(y)) -Ejv(-log/^(iV)) 

= Ex,w (- log fy (;^AX + ^) ) - ^n{- log fN{N)) 

frlzh^X + N 



^^--1-^°^ /.(AT) ^ 1- (111) 



But clearly /y(y) = E.IPx(x)/^(y - ;^Ax), and thus 



I{X;Y)=Ex,n\ -log 



>Ejv -logE 



X 



fN{N) 



E^ -log Y.^x{x)Pxix') ^^" „ ,^, ^ . (112) 



Now let ^ be a random matrix with i.i.d. entries. By taking expectation with respect to A, we get 

/ / fN(^A{x' -x) + n\ 

E^(I(X;y)) >E^,^ -log 2^''(^-).^Px(.)Px(xO ^g,(^,)^^(^) ^ 



/ X / fNi^Aix' -x) + N)\ 

Using the Jensen's inequality again, we will have 

EAa(X;y))>-mE(g(Ari))-log^Px(x)Px(xOEjv,A 2E.(^0/^(Ar) ' (H^) 

Now, note that /at {x) = HjLi /a^, (2;^) = ]Xj=i f (xj) and hence 

fN {^A {x'-x) + n) _^ f {^A jx' - x)^ + N,) 
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For fixed x and x' , A{x' — x)- and Nj are independent for j = 1, . . . , m. 
Thus, the expectation and product operators commute; we then get 



""■4 .?,i.,,..,m M =n^ 



N,,A 



2EgW)/^(Ar) I 11 ^^-^ I 29^)/ (at. 



i=i 



j^ 



'^^uA I 2'?W)/(iVi) ' ' • ^^^^^ 



But 



^^^ I 2.(^0/ (AT,) j = 7 ^iR7(^I^ ^(^)^^ 

fJ^A {x' -x)^ + u) 2-«(«)dn = / / (n) 2-<"-^^("'-")0dn 

= Ejv, (2-<'^^-^^("'-")0y (117) 

After substituting in the previous relations, we get 

EAmX;Y)) > -mE(<?(iVi))-log j J]Px(x)Px(a^')(EiV„A(2-''('^^"^^^^"""^^^) 



m\ 



-mE{q{Ni)) - log ( E^( E^,,,( 2 "^""^ -^ J ] ] ). (118) 



Now since A = rB, we get 

E^(I(X;y))>-mE((7(iVi))-log('E^('E^,,, ('2-^('^^-'^)')') V (119) 

The right-hand side of the above inequality is a lower bound for the average mutual information where 
A and X are chosen at random with appropriate probability distributions. The choice of r guarantees the 
SNR constraint on the average for this class of random matrices. Since the capacity is the maximization 
over 7r(-) and p{-), the inequality holds. D 

Note that although in the proof of the above theorem, the SNR condition is only satisfied on the average, 
because of the measure concentration phenomenon [13], almost all matrices (w.r.t P'^) result in almost 
the same power as well as almost the same mutual information. Thus, a typical matrix simultaneously 
satisfies both the SNR condition and the bound for mutual information with high probability. 

To obtain a true bound for very small values of m and n, one can start from the equation 

Ea{I{X;Y))=F{Q)xKa{I{X;Y)\Q) + {1-F{Q))xEa{I{X;Y)\Q^), (120) 
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where Q denotes the event that the (random) matrix A satisfies the SNR condition. If |X| = q, then 

1 



Ea{1{X;Y)\Q)> 



■(Q) 



{EA{I{X;Y))-{l-¥{Q))nlogq). 



Now, it remains to evaluate a lower bound for P(Q). But 



P(Q) = 1-P a^tr (BB*) + ulti (BJB*) > 



nrjaim 



(121) 



(122) 



which can be controlled using the Markov inequality in the general case or using Chemoff like inequalities 
if /ip = since tr (BB*) is the summation of i.i.d. random variables. 



J. Proof of Corollary^ General Lower Bound for the Sum Capacity for (q, l)-ary CDMA Systems 
Suppose X has /cq number of O's and ki number of ij's. The number of such x's is 



ko\ki\...kii\ 



with 



probability n?=oi'j • ^"'^ ?>\ich x's, suppose Uij is the number of Siij terms in the product 6^x; thus 
uij + U2j + ■ ■ ■ + uij = kj and if Uj/s are fixed numbers, the number of corresponding b'^ vectors is 
nti .,j.^L.»,.! with probability Hli^t^'-"'"- 
Therefore, we have 



E, 



1 + 7 



\ 



E n 

i<i<9 



Uii,...,Uu 



n 

t=i 



TTt' 



ELi-t.e5(T^fy;;rlEU«.(E|.x".. 



(123) 



D 



K. Proof of Corollary [2 Results for Finite User Inputs and Symmetric Signature Matrices- Noisy Case 



Suppose X has ko number of O's and ki number of Ib^j's. One can easily see that from the symmetric 
distribution on the signature alphabets, the expected value on the vector b remains constant; i.e., ±ii 
behaves similar to the single symbol +z~j. The rest of the proof is the same as Corollary |6] The second 
part is exactly the same as the first part and is straightforward. D 
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L. Proof of Theorem \22\ Lower Bound for Binary Input and Real/Complex Valued Signature Matrix 

We use Corollary |5] where 7r(-) is the distribution of a standard Gaussian real r.v. for the real case, 
or the distribution of a Gaussian complex r.v. with independent standard Gaussian coordinates for the 
complex case. In this case, X is a vector of i.i.d. entries with values I = {iq, ib«i, ib«2, • • • , ±^5'}- By 
noting that the distribution is symmetric and Gaussian, it is easy to see that if the number of itzj entries 
of X is fixed, then the expected value on the vector b remains constant. Let ki be the number of itzj 

entries of x; the number of such x's is yWVTT^.'^^^^^'^^ '^^^' ^^'■'^ probability n?=o^?- ^^^ ^^ ^ Unear 
combination of independent Gaussian random variables, which is again Gaussian. By using the fact that 



^ 21^ — 2 for the real case and r = . I 2_P^ — 2 for the complex case, (l63l ) and (l64l) are easily 
derived. 

M. Proof of Theorem \23\ A General Upper Bound for the Noisy Case 

Our conjecture is based on i.i.d. user input distribution. Note that 

I{X;Y) = M{Y) - M(Y\X) = EI(y) - El(iV) = m{Yi,Y2, . . . ,Ym) -mH(iVi) < m(H(yi) - H(A^i)). 
When S = {si, S2, ■ ■ ■ , Sq}, M.{Yi) is maximum when we have Ui number of Si in vector Yi. Let p be the 
i.i.d. product distribution which maximizes I{X;Y); thus, C{m,n,I,S,rj) < nM.{Z). Suppose Vij is the 
number of Sitj terms in Yi = -^ Y17=i '^li^i + ^; the number of corresponding x's is nA:=i iv^i v ! 
with probabiUty Yl1=iPk^°=^^''''- Therefore, Yi has the following distribution: in which 

/» = E fri L, "' ., )] (iip^^'-'A /(- -^(E^^e--^-))' (124) 



where 




fc=l / " k=l a=l 



V -nix^+y^) 



f{x + V-ly) = -^e-'^^^+y > (125) 

vr 

is the distribution of a complex Gaussian. 

Also, when Sj = e~i~v~^ and / divides n, we conjecture that ui = U2 = ■ ■ ■ = ui = j. . D 

A^. Proof of Theorem \24\ Asymptotic Lower Bound for Finite Input and Additive Gaussian Noise 



lira — 

m,n— >oo ji 



-rwy log e — log E^ E;, 




1 + 7 



1 1 [lie 2(7+7) I 'v^ I' , , 
--7loge- lim -logE^ E,, — . (126) 

jj n-s-oo n \ \ 1 + 7 III 
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Now, let n be a large number and X = x; name the empirical probability measure induced by x on 



I as p (i.e., a proportion of p (i) of entries of x are i). In this case, E ( ^-^ I = ^/n^^^^p. Therefore 



- — ^^ — - — is exponentially small and negligible. When /^Tr^p = 0, according 
to central limit theorem —7^ is a complex Gaussian random variable Z. Thus, we have: 

£2(1 + 7)1 %/S I \ / g(^p^+r.Mp'-^)(^x^+Mx^) 1+7 \ 

1 + ^ I - ^ (^ TT^^ j = ^^^^- (127) 

If //p / 0, the above expression approaches -^7- which yields a trivial bound. Hence, we assume that 
[ly = 0; thus we have 



TERM = I :r^—^ ,„ , , (128) 

where Ai,A2 are eigenvalues of the covariance matrix of an r.v. which has the distribution of the product 
of two independent variables with distribution p and vr, and hence 



E, I "" ' « 2^"°'^^^+^+'^"^ (.p-)(.:V..^) )+log(l+7+2/3>?7 (.^2)(;^2,,^2) )] (129) 

On the other hand, by Sanov theorem [56] about the large deviation probability of empirical measures, 
the probability of appearance of p is asymptotically equal to 2~"'^^p"p>, where D(p||p) is the KuUback- 
Leibler distance of p and p. Thus, we have 

/ 



-^7log(e)-^ 



7log(e) - -logE^ 



V 




£2(1 + 7)1 V^ ' \ \ 1 1 



sup 

p 



loge A / ^ 2^!?2^i \ f,^ ^ 2/3r?7A2 

' log 1 + 7 + „, „ , — ^ + log 1 + 7 + 



(130) 



From the above equation and (11261 ) , we get the desired result. D 

O. Proof of Active User Theorems 

1) Proof of Theorem \25\ Capacity Lower Bound for Binary/Real Active System: 
We use Corollary |5] where vr (•) is an J\f (0,1) random variable and p{-) is uniform distribution on 
{±1}. We have X = X — X' where X and X' are two A;-active vectors. We evaluate the expectation 
conditioned on ||X|p = 2i for some i in the interval [0,2k]. Assume that the corresponding entries in 

X and X are of the form (+1, -1) , (-1, +1) , (-1, -1) , (+1, +1) , (0, -1) , (0, +1) , (+1, +1) , (-1, 0) 

~ 2 
, (0,0) ki,k2, ■ ■ ■ ,kQ times, respectively. Now in order to satisfy the condition ||X|| = 2i, we require 
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the following equations: 

Ej ^i - (^7 + k8 + kg) = k 

Y,ki = n 

Y,ki- (fcs + kQ + k<j) = k 

Aki +Ak2 + k5 + kQ + k7 + k8 = m = 2i € [0, Ak] 

The simplification of the above equations lead to: 

ki + k-2= j 
k^ + ki = k-i + j 
k^ + kQ = i- 2j 
k7 + k8 = i- 2j 
kg = n + 2j — k — i 
The probability that the above equation are satisfied simultaneously is equal to — vj.^+j .,. 2J,,^ kj 

and the expectation term in ( [59l ) is ( 1 + ^ A""^ k ) ^ . By varying the value of i in [0, 2k], we get the 



mfe(l+7) J 

desired result. D 



2) Proof of Theorem \26\ Lower Bound for k- Active Real/Real CDMA Systems: 
Let 7r(-) be the standard Gaussian distribution on M. According to Corollary |5l we have 

EB(I(X;y))>-m7loge-logE^ eJ^^^^^^^^- || |, (131) 

6'^X~ \X\Z and hence 




E;,(e2(i+-')'"' ' ) = E ( eV 2(1+^)" ^ ) = , . (132) 

^ ^ (l+7)m 



Thus, 



¥.B (I {X- Y)) > -m (7 log e - log (1 + 7)) - log E^ I I 1 + .f ^'^'^ J ), (133) 

where r = ./-^ and X = X — X' is the difference of two independent fc-active vectors. Conditioning 
on the number of common non-zero elements of X and X' to be j, one has \\X\\'^ ~ 'X^ik-j) + ^^? ^'^'^ 
therefore, we derive (1761 ). D 
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3) Proof of Theorem\27\ Capacity Lower Bound for Binary/Binary Active System: 
We use Corollary |5] where vr (•) and p{-) are uniform distributions on {±1}. We evaluate the expectation 
conditioned on the vector X to have i elements in {±2} and j elements in {±1}. Obviously < i+| < A; 

j 

and j € 2Z, the probability of this condition is equal to ^/, a (■ . j . _ "j _. _ ) and the expectation 
term in ( [59l ) becomes 

/ 1 



.^^'■^■'Sw + ^-2^ >''"''"'' 



Now by averaging over all possible values of i and j we get dTT] ). 



(134) 



□ 



4j Proof of Theorem |2S] Active Users with Probability pact, I = S 



From Corollary [5] for any realization of X, we have b^X r^ AfiO, \X\ ) and hence 



,~,,2 



Ef, e2(i+^)' 



-Ife^xr 



1 + 



(l+7)m 



In the Pact-active case, 



therefore: 



/ 2r/ 




(135) 



Cactive("i,?T',M,IR,??,Pact) > SUp{-m(7loge-log(l +7)) -logE^ 1 + 



(1 + 7) mpact 



(136) 



}(137) 



Now X = X—X', where X and X' are independent with i.i.d. entries which are M (0, 1) with probability 
Pact and with probability 1 — pact- Hence the entries of X are 

with probabiUty (1 — Pact)'^ 

AT (0, 1) with probabiUty 2pact (1 - Pact) (138) 

TV (0, 2) with probabiUty pl^^ 

Now conditioning on the number of J\f (0, 2) and J\f (0, 1) elements of X to be ki and k2, respectively 
with the probability 



n 



ki,k2,n — ki — A;- 



pi';l{2pact{l-Pact)ri'i^-Pact 



,2(n— fci— fc2) 



,%,,2 



We have ||X|| ~ ^Xki +Xk2 and thus we arrive at ( 1791 



(139) 

D 



67 



5) Proof of Theorem\29\ Asymptotic Lower Bound for a Real/Real pact-^ctive CDMA System: 
From Corollary |5] and (I136I ). we have 

2 \ -m/2^ 



/(X;y)>-m[7loge-log(l + 7)]-logE^ I (l + ^:p^i^-l— ) |, (140) 



^l + 7)mpact 



where X = X — X'. We assume that elements of X or X' are zero with probability 1 —pact and AA (0, 1) 
with probability pact- Now we have, 



lim -I(X;y)> ^(7loge-log(l + 7))- lim -logEl(l+, , , 

n/m—^P n/m—^P \ \ / 



2iv\xf \ 




= -^(7 lege -log (1 + 7))- lim i logE ( 2"-^( - ^ ,(141) 
where F{x) = -^ log (l + q , T'^ — x\. Now by Varadhan's lemma [57], the last expression is equal to 

II X'll^ 

supj.. {^F{x) — lix)], where l{x) is the rate function for the sequence of random variables ^^ — ^^ But 

~ 2 

||X|| is the sum of n i.i.d. random variables Yi(\ < i < n) oi the form 

(1 - Pactf , 

Y.r^iz' 2p,M - p,,,), Z ~ Ar(0, 1). (142) 

The Cramer's theorem [57] gives the rate function as: 

/(x) = sup {log ex xt- log M(t)}, (143) 

t 

where 



M{t) = E(e*^') = (1 -pact)'E(eO) + 2pact(l - Pact)E(e*^ j +pl,W.(e 

ict(l ~ Pact) 



(1 -Pact)' + ^P-^(iZ^-') + ^k^. (144) 



D 



6) Proof of Theorem \31} Binary Input and Binary Signature (p-Active): 
We use Corollary [5] where 7r(-) is the uniformly distributed on {±1} and p(-) is uniform distribution on 
{±1}. In this case X is a vector of i.i.d. entries with values in {0, ±1, ±2}, where p{0) = (1 — pact) + ^ 
and j)(±l) = 2pact(l — Pact) and iJ(±2) = 2|!_. Now, applying the condition on vector X to have ki 
elements in {±2} and k2 elements in {±1}, we have < ki + k2 < n. However the probability of 
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this condition is equal to 2^^-'^i(^^ ^^ „" fci-fc^)^^':'^''''^'''!^ + i^^'^'^ ~ 2pact)"~^'~'''- The distribution of 
b'^X is equal to the sum of ki unifonn distribution on {±2} and k2 uniform distribution on {±1}. 
we can assume that i element of these ki elements are {+2} and j element of these k2 elements are 
{+1}. Therefore, the conditional expectation is equal to 




Now by varying for all possible values of ki and k2, the theorem is derived. 
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